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A GLIMPSE OF THE FUTURE 


Theodore Lewis 

Professor, Department of Work and Human Resource Education 
University of Minnesota 

As part of the usual end of year meeting that has been a feature of our NCETE project we 
organized a conference intending to provide a forum for young voices in an outside of the 
Center. The basic model for the gathering was borrowed from track and field. This was to 
be an intellectual conference meet, to which teams of scholars and their coaches/mentors 
would come, drawn from university programs where scholars worked at the intersection 
of engineering education and technology education, to share ideas and to engage each 
other. The conference theme was to be Research in Engineering and Technology 
Education (RETE). The invitation to program leaders asked for the following: 

a. Reports on recently completed or partially completed dissertations 

b. Presentations of doctoral research proposals 

c. Presentations of issues/theory/challenges that have bearing on RETE. 

d. Reports from the field (e.g. Some students may have been working with 

schools, either teaching or engaged with professional development) 

On the conference morning a regular campus bus drove up, having just crossed the 
Mississippi River that divides the main campus of University of Minnesota, and onto it 
climbed a quite remarkable gathering of talent, young as well as seasoned, all anxious 
and excited, and looking forward to the day’s proceedings that would begin at the end of 
the short trip to the conference venue. Represented in this gathering were doctoral teams 
from Utah State University, The Ohio State University, Virginia Tech, Purdue, Tufts, 
University of Illinois at Urbana-Champaign, University of Georgia, Colorado State, and 
University of Minnesota, the host. Equally, this gathering included representatives from 
arguably the most significant Technology Teacher Education programs in the U.S., 
including Illinois State University, University of Wisconsin-Stout, California State 
University, Eos Angeles, North Carolina A & T State University, and Brigham Young 
University. Christine Hailey, Director of NCETE, welcomed the gathering, which 
included Karen Zuga, NSE Project Director with responsibilities for the Center. And the 
program then began with Dr. David Strieker, a Eellow from the first NCETE cohort, 
having the honor of leading off with a presentation of findings from his recently 
completed dissertation. 

These proceedings summarize ideas from the presentations made during the day. The 
reader will notice that there was clustering of ideas around themes. One group of papers 
focused upon exploring teachers ’ knowledge and dispositions to STEM curriculum and 
instruction. These can be seen in the contributions of Jenny Daugherty, Todd Eantz, Ered 
Eigliano, Brent Holt, Morgan Hynes, and Todd Kelley. A second set of papers focused 
upon dealing with ill-defined problems and complex systems. These can be seen in the 
contributions of Ben Eranske, Steven Rigby, and Douglas Walrath. 



A third set of papers examine problem solving predisposition in children and adolescents. 
We see the issues here in the work of Katrina Cox, Michael Nehring, and Leah Roue. 

Cox looks at the effects of me tacognitive journaling, an approach that is significant given 
the important role that journaling plays in the practice of engineering. Nehring and Roue 
examine fundamental developmental issues attending the pre-disposition to design, 
touching upon critical notions such as design intention, design stance, and functional 
fixedness. Shawn Gordon and Nathan Mentzer explore the impact of engineering design 
experiences on student motivation and learning. Gordon’s approach was to have students 
build Rube Goldberg machines. Mentzer’s approach was to engage students in design 
challenges. 

Three of the papers explore student interest in STEM careers. Chandra Austin and 
Cameron Denson address the knotty question of African American students self-efficacy 
beliefs regarding engineering careers. Denson has explored whether mentoring can be a 
way to enhance the beliefs of these students such that engineering careers can be 
attainable for them. Mark Mahoney examines whether a special Metro school (in 
Columbus, Ohio) intended to enhance student interest in STEM-related actually achieves 
its purpose. 

Beyond papers that cluster thematically are a few that pursue interesting lines that were 
influenced by cognitive and creativity considerations. They included Brian Gravel’s 
examination of children’s representation of scientific ideas; Matthew Lammi’s 
examination of differences in the cognitive strategies and processes of expert and novice 
problem solvers; and David Striker’s comparison of the perceptions of creativity held by 
art, music and technology teachers. 

What can’t be captured in this introduction to these proceedings is the electricity of the 
conference. Those of us from earlier eras developed the consensus view that what we saw 
and heard was incomparable in our experiences as technology educators. We saw 
doctoral students drawing upon theory as they framed their studies. We saw a strong 
focus on cognitive science and creativity ideas. Several of the presenters situated their 
work in classrooms and resorted to Vygtosky and Piaget as they tried to unravel what 
they saw. Then there were doctoral students who focused their sights on teachers— on 
their professional development needs, and on their pedagogical insights. Several students 
looked more closely at problem solving and design, in ways that suggest that, contrary to 
a prevailing view in technology education, these really are unsettled notions, rich in 
inquiry possibilities. 

We old timers had caught a glimpse of the future of the field, with a completely new set 
of issues; a new set of conceptual frameworks; a new synergy derived from the 
interaction between engineers, engineering educators and technology educators; new 
institutions (such as Tufts and Purdue) coming into the mix; young women and minorities 
holding their own; and overall just the most delightful assemblage of young talent one 
could hope to be heirs to a field. 
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Engineering byDesign™ PROFESSIONAL DEVELOPMENT 
EVALUATION SUMMARY REPORT 

Jenny Daugherty 

University of Illinois at Urbana- Champaign 


Abstract 

Through the Engineering byDesign™ (EbD) eurriculum projeet, professional 
development workshops and an online tool, eTIDEonline, were developed to ereate 
resources, provide training, and serve as a collaborative environment for EbD teachers. 
The EbD program is facilitated by the Center to Advance the Teaching of Technology 
and Science (CATTS), which is the professional development arm of the International 
Technology Education Association (ITEA). The EbD program is the only standards- 
based national curriculum model for grades K-12 that delivers technological literacy. In 
addition to curriculum development, the EbD program also concentrates its efforts into 
professional development. The goals of the professional development efforts of CATTS 
are to encourage and prepare teachers to implement the EbD curriculum and to develop a 
community of learners. 

The second year of EbD professional development was undertaken at the 2007 
International Technology Education Association’s conference in San Antonio, Texas. 
The EbD professional development focused on two of the EbD courses: (1) Engineering 
Design and (2) Exploring Technology. This evaluation aimed to include the entire 
population of teachers, superintendents, or teacher educators involved in the second year 
of EbD professional development. The first phase of the evaluation, consisting of an 
online survey, gauged the value of the professional development efforts as perceived by 
the teachers and estimated their preparedness in implementing the curriculum. The 
second phase of the evaluation, consisting of a similar online survey, estimated the 
implementation of the EbD curriculum, challenges encountered by the teachers not 
addressed by the professional development efforts, and participation in eTIDEonline. 

Research Questions 

The following key questions guided this evaluation: 

1 . How well did the workshop help prepare teachers to teach the curriculum? 

2. What are the strengths of the professional development efforts (the workshop and 
eTIDEonline)? 

3. What aspects of the professional development efforts could be improved? 
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Evaluation Design 

The overall approaeh that was utilized in this evaluation was a mix of a behavioral 
objectivist approach and a utilization-focused approach. The objectives of the 
professional development efforts have been clearly prescribed, including: (1) to prepare 
teachers to teach the curriculum, and (2) to serve as a collaborative environment for EbD 
teachers. The evaluation focused on providing data to gauge how the professional 
development is meeting these two objectives. In addition, this type of approach provides 
formative data to inform the decision-making processes of the primary stakeholders. This 
evaluation is focused on an intended use (changes to the professional development 
efforts) by the intended user (the CATTS director). 

Data Analysis 

The response rate for Survey 1 was 86%, with 24 of the 28 participants responding to the 
online survey. The response rate for Survey 2 was much lower (57%) with only 16 out of 
the 28 participants responding. The surveys were analyzed according to the type of data 
collected. Item I on both surveys provided nominal data so that frequency counts could 
be provided on which course the participant planed to teach or was teaching. For items 2 
through 5 on Survey 1 and items 2 through 4 on Survey 2 the Likert scale data were 
treated as continuous variables from which descriptive statistics were analyzed. For 
section three of the survey (items 6-8 on Survey 1 and items 5-7 on Survey 2) qualitative 
data were analyzed using a method called content analysis. Themes and patterns in the 
data were derived, coded, and categorized in an effort to understand and explain the 
specific elements each question sought to answer. 

Findings - Planned to Actual Implementation 

There was drop between those who planned on implementing an EbD course (18) and 
those who were currently teaching one at the time of the second survey (13). Eight 
respondents planned on teaching the Engineering Design course, however, none of the 
respondents to the second survey were currently teaching this course. Although this 
decrease in planned to actual implementation cannot be attributed solely to the 
professional development efforts, it is important to note this finding particularly in 
regards to the Engineering Design course. 

Research question one asked how well the workshop helped prepare teachers to teach the 
curriculum. Based on the majority of responses in survey 1, the workshop helped to some 
extent (33.3%). However, 25% of the respondents felt that they were prepared to a great 
extent. This finding is consistent with the results from survey 2 where 43.7% of the 
respondents felt they were prepared to some extent. 18.7% felt prepared to a great extent. 
When asked whether the workshop answered all of their questions, the respondents 
largely indicated that it did not or only to a little extent (45.8%). Only 29.2% said it did to 
some extent. However, when asked if the workshop was worth attending, 29.2% said it 
was to a great extent and 25% to some extent. 


4 



The second key question driving this evaluation aimed to distill the strengths of the EbD 
professional development efforts. With most of the participants (41.7%) indicating that 
the workshop was worth attending to a great extent or to a very great extent, the 
participants seemed to believe that the workshop was beneficial. When asked to describe 
the ways the workshop has provided the necessary information about the curriculum the 
general themes that emerged were that it: (a) provided an overview of the curriculum, (b) 
provided a few examples of student work and activities, and (c) made them aware of the 
goals and aims of the curriculum. 

Exploring the weaknesses of the professional development efforts was the intent of the 
third key question. When asked to indicate the questions that respondents still had about 
teaching the curriculum, questions centered around three themes: (a) planning, (b) 
implementation, and (c) course content. Respondents also provided information 
concerning the changes they would like to see made to the workshop. One respondent 
indicated that he or she would like to see more specifics on the lesson plans and how to 
implement. Another would have liked to have seen the cTIDEonline environment. In 
regards to what the respondents felt they needed to teach the curriculum, the responses 
varied from more resources to more in-depth training. When respondents were asked 
what additional mathematical and science concepts were needed to implement the 
curriculum, responses also varied from none to a tremendous amount. One respondent 
stated that technology education teachers “do not have the background to teach this.” 

eTIDEonline 

Eocusing on the eTIDEonline component of the EbD professional development, the 
findings from both surveys indicated a desire to use the site but some obstacles to 
actually using the site. When asked in the first survey whether respondents intended to 
use the online tool, 16.7% responded to a very great extent, 37.5% indicated to a great 
extent, and 29.2% responded to some extent. When asked in the second survey whether 
respondents were participating in eTIDEonline, the responses decreased with 0% 
participating to a very great extent, 12.5% participating to a great extent, and 37.5% to 
some extent. 18.7 % were not participating at all. Asked whether respondents believed 
the information posted on the site to be helpful, 31.3% of the respondents indicated that it 
was to a great extent. When asked what would increase their use of the online tool 
responses included the increased participation by other teachers, more time, more 
participation by those piloting the curriculum, and more information to use with students. 

Recommendations 

When asked what advice they would give to CATTS, numerous responses were provided. 
A few of the respondents recommended that the professional development involve an 
intense examination of a unit or lesson. Another suggestion was to provide a more 
extensive experience with a week-long program. More structured training with more 
hands-on activities and examples was also a prominent suggestion. In addition to these 
recommendations a few other suggestions based on the professional development 
literature are outlined below. 
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As Bybee and Loucks-Horsely (2000) articulated, four key components are necessary for 
effective technology education teacher professional development: (a) teachers need to 
develop skills related to technology; (b) teachers need opportunities to learn about how to 
teach technology; (c) teachers need tools and motivation to continue their own learning; 
and (d) long-term professional development is required to support the changes required 
for the STL to be successful. In 2001 Bybee, put forth design principles for effective 
professional development of technology education teachers, including: 

1 . Student learning should be at the core; 

2. Technology education pedagogical content knowledge should be developed; 

3. Student learning principles should guide teacher learning; 

4. Learners’ current understandings should be acknowledged; and 

5. Professional development must align with and support system-based changes. 

Despite the lack of research on how these characteristics improve teaching and student 
learning, a consensus has emerged within the literature about the characteristics that 
differentiate “high quality” professional development (Desimone, Porter, Caret, Yoon, & 
Birman, 2002). Loucks-Horsely (1999) identified four clusters of variables that affect the 
quality of professional development including: (a) content; (b) process; (c) strategies and 
structures; and (c) context. Based on these clusters and respondents’ suggestions, the 
following recommendations were provided to improve the EbD professional 
development. 

Recommendation 1 - Content 

• Focus the PD workshop on a particular lesson or activity and allow participants to 
work through the activity. 

• Spend time educating participants on both the structure and goals of the 
curriculum and on the particular content knowledge needed to better implement 
the curriculum (i.e., math and science concepts). 

Recommendation 2 - Processes 

• Guide participants through an activity or unit from the perspective of a student 
and then through the perspective of a teacher. 

• Offer deliverables from the workshop - completed activities, examples of student 
work, lesson hand-outs. 

• Demonstrate how to login, post, and best utilize the cTIDEonline environment. 

• Continue professional development experiences through eTIDEonline by 
scheduling online meetings with the PD facilitator to answer questions; delivering 
added content knowledge; providing support materials to implement particular 
activities, etc. 
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Recommendation 3 - Strategies and Structures 

• Publicize workshops facilitated by curriculum specialists more broadly on EbD’s 
website. 

• Send eurriculum specialists to local and state technology and engineering 
education conferences to present the curriculum. 

• Coordinate a week-long summer PD experience that draws teaehers nation-wide 
and is facilitated by the eurriculum specialists. 

• Expand the scope of the eXIDEonline tool to serve as a support to teachers by 
providing aceess to program direetor, currieulum developers, currieulum 
speeialists, and field test teaehers 

Recommendation 4 - Context 

• Market ITEA conferenee workshop as an informational session and strongly 
suggest to participants to attend a workshop facilitated by a eurrieulum speeialist. 

• Develop cohorts of teachers who first attend an informational session, then a 
professional development workshop, and eontinue collaborating on eXIDEonline. 
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KNOWING WHAT ENGINEERING AND TECHNOLOGY TEACHERS NEED 
TO KNOW: A CONTENT ANALYSIS OF PRE-SERVICE TEACHERS’ 
ENGINEERING DESIGN PROBLEMS 

Todd D. Fantz 
Colorado State University 


Abstract 

According to the American Society for Engineering Edueation, the four STEM 
disciplines should not be taught in isolation in a sehool curriculum, but interdisciplinary 
reinforced, and continually cross referenced, as part of a dynamic triangle that ultimately 
researches, designs, and ereates the way we live, work and play. Therefore, it only makes 
solid aeademic and professional sense to prepare teaehers who are highly qualified to 
deliver this integrated content in a K- 12 setting. This is a powerful, yet eritical, void in 
most publie education and professional teaeher preparation programs who desire to 
respond to the call for introducing engineering coneepts into the professional teaeher 
preparation programs that train teehnology education teachers. 

Signifieant questions exist around the requisite content knowledge required of a 
teehnology teacher to infuse valid engineering concepts into the K-12 classroom. What 
are the appropriate mathematieal and analytieal levels required of pre-serviee teacher 
preparation? Can engineering trained pre-service professionals deliver instruction and 
teach engineering design lessons that are eontent and context valid? These and many 
more questions remain as the teehnology teacher preparation community begins to join 
with other key stakeholders in preparing teaehers to respond to the national call for a 
stronger engineering STEM emphasis in K-12 edueation. 

This paper will report on content analysis researeh of pre-serviee teaeher/engineering 
seienee students’ ability to conceptualize, design, and evaluate student design brief 
solutions in high school technology classes. This researeh is just a part of a larger 
challenge within the engineering and technology teacher preparation community to 
understand what pre-serviee teacher candidates need to know and be able to do to teaeh 
engineering design in a eontext and content valid manner. This report on the student 
instructional design content analysis will use a quantitative coding scheme that maps the 
design brief problem elements and student solutions to the engineering design process. 

Introduction 

Since its evolution from industrial arts, technology education has struggled to move from 
a eurriculum based on human productive praetiee to a legitimate general school subject 
focused on teehnological literacy for all students. Due to this struggle, industrial arts and 
indeed technology education has undergone changes in its name, eurriculum, 
instructional strategies, materials, and technical content many times over (Eewis, 2005). 
Some previous names of industrial arts and teehnology education curriculum and content 
framing efforts inelude the Industrial Arts Currieulum Project, Maryland Plan, Jackson’s 
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Mill, and Technology for All Americans Project (Hill, 2006). The current movement in 
the turn to engineering involves incorporating engineering design as a focal point for 
technology education. Significant questions exist around the requisite content knowledge 
required of a pre-service technology teacher to infuse valid engineering concepts into the 
K-12 classroom. What are the appropriate mathematical and analytical levels required of 
pre-service teacher preparation? Can a professional not trained in the engineering 
sciences deliver instruction and teach engineering design lessons that are valid in content 
and context? These and many more questions remain as the engineering and technology 
teacher preparation community begins to join with other key stakeholders in preparing 
teachers to respond to the national call for a stronger STEM emphasis in K-12 education. 

Why Engineering Design and Why Now? 

With the high-tech boom of the 1990’s, the bust of 2000-01, and the ensuing anxieties 
about competitiveness and national security, K-12 STEM education has commanded the 
attention of people far beyond the community of educators typically involved in the field. 
Policy-makers, industry leaders, and thoughtful leaders in the media have registered 
concerns and volunteered solutions regarding apparent problems with how science and 
mathematics are taught and learned and how technology figures as both a feature of these 
topics and a downstream result of their inculcation. All the while, engineering and 
technology has largely remained a shadowy presence in discussions about the K-12 
STEM education, a spectral “T&E” quietly inserted among its more concrete 
complements, yielding, if nothing else, an acronym that lends itself nicely to speech and 
writing. In addition, the technology education community must be cautious in assuming 
that the “T” in STEM refers to technology education. General interpreted in the 
educational and professional community could reserve the “T” in STEM for information 
science and computing technology and not technology education. In K-12 schools this 
could mean educational technology. 

Recent arguments in favor of defining and implementing a more substantial role for 
engineering in K-12 STEM education are many and strong. Starting with the most 
general, K-12 education, as a democratic institution, should provide meaningful 
preparation for its graduates, in all their representative diversity, to participate fully in the 
opportunities available to them in society. Among the ways that people participate in 
society — at home, at work, in communities, or any other context — almost every 
experience is shaped by a product or environment that results from engineering. 
Buildings, clothes, cars, clean water, indoor climate control, personal technologies, and 
nearly everything else people encounter in daily life comes from engineering. An 
education system that treats this area of activity obliquely, if at all, is failing to prepare 
students for the world they will enter upon completing their studies. Understanding first 
that the world they inhabit is engineered and second that the engineered world takes the 
shape it does through human choice and activity, areas in which they themselves can 
participate, is a fundamental precondition for full participation in twenty-first century 
life. 
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Engineering is an important national resouree in efforts to keep Ameriea eompetitive in a 
knowledge- and teehnology-driven global eeonomy and safe in an uneertain geopolitieal 
elimate (National Researeh Couneil, 2006). Technologieal innovations result from the 
work of people trained in engineering and technology fields. Educated across disparate 
areas of science and mathematics, these people translate their understanding of 
fundamental science and mathematics into usable objects and applications that improve 
our lives, create new jobs and industries, and extend the frontiers of human possibility. 
They also play a fundamental role in national security strategies, combating threats to a 
country’s citizenry through research and development of technologies that can neutralize 
threats to civilian and military populations. 

Engineering conveys practical, classroom benefits for educators and students, as well. A 
way to bring to life sometimes abstract, difficult topics in math and science, engineering 
can make the classroom exciting and relevant to lived experience. Research shows the 
integrative, applied nature of engineering can enhance student learning, boosting test 
scores and helping schools meet standards-driven education requirements (Baker 2005). 
The collaborative, socially beneficial aspects of engineering have also been shown to 
appeal to students whom the field has traditionally failed to engage, including females 
and under-represented minorities (Guertin and Rufo 2004, Wiest 2004). 

Do Technology Education Pre-Service Teachers Have the Pedagogical Content 
Knowledge to Teach Engineering Design? 

Geddis (1993) described pedagogical content knowledge (PCK) as a set of attributes that 
helped someone transfer the knowledge of content to others. According to Shulman it 
includes "most useful forms of representation of these ideas, the most powerful analogies, 
illustrations, examples, explanations, and demonstrations-in a word, the ways of 
representing and formulating the subject that make it comprehensible to others" 

(Shulman, 1987, p. 9). 

In addition, Shulman (1987) suggests that PCK is made up of the attributes a teacher 
possess that help her/him guide students towards an understanding of specific content 
such as engineering in a manner that is meaningful. Shulman argued that PCK included 
"an understanding of how particular topics, problems, or issues are organized, presented, 
and adapted to the diverse interests and abilities of learners, and presented for 
instruction" (1987, p. 8). In light of what engineering and technology education teachers 
should know and be able to do, Shulman argued that pedagogical content knowledge was 
the best knowledge base of teaching and suggests the key to distinguishing the 
knowledge base of teaching lies at the intersection of content and pedagogy, in the 
capacity of a teacher to transform the content knowledge he or she possesses into forms 
that are pedagogically powerful and yet adaptive to the variations in ability and 
background presented by the students (p. 15). Therefore, the intersection of engineering 
science, i.e., knowledge of the scientific and mathematical knowledge needed to engage 
in the analytical aspects of design, knowledge of engineering design, with teaching in the 
technology education classroom will wholly depend on the ability of teacher educators 


10 



and pre-service teachers to transform this knowledge into adaptive instruction, with 
which students can engage. 

The Design Processes 

The underlying distinction between technology education and engineering lies with the 
design process. Hailey, Erekson, Becker, and Thomas, (2005) developed a comparison 
table of the design processes for the two disciplines .Table 1 displays the side-by-side 
comparison between an engineering design process and a technology education design 
process as listed in their manuscript. 

Table 1. 


Design Process Comparison 


Engineering Design Process (Eide, Jenison, 
Mashaw, & Northup, 1997) 

Technology Education Design Process 
(Standards for Technological Eiteracy, 
2000) 

1 . Identify the need 

1 . Defining the problem 

2. Define the problem 

2. Brainstorming 

3. Search for solutions 

3. Researching and generating ideas 

4. Identify constraints 

4. Identifying criteria 

5. Specify evaluation criteria 

5. Specifying constraints 

6. Generate alternate solutions 

6. Exploring possibilities 

7. Engineering analysis (applications of 
math and science) 

7. Select an approach 

8. Optimization 

8. Develop a design proposal 

9. Decision 

9. Building a prototype or model 

10. Design specifications 

10. Testing and evaluating the design 

1 1 . Communication 

1 1 . Refining the design 

12. Make it - create it 

13. Communicating results 


Whereas, the design processes of engineering and technology may look similar, there are 
distinct differences. While the technology design process is more concentrated on the 
building and testing aspects, the engineering design process is more focused with analysis 
of design. To add to this discrepancy. Hill (2006) alleged practicing technology educators 
do not use the steps of “identifying criteria” and “specifying constraints.” In the 
beginning, both processes identify what the need or problem is, and who will benefit 
from a solution. Both processes also search for solutions from various sources and 
identify what restrictions will be placed on the design. However, the next step is 
drastically different between the two. The technology design process jumps to selecting 
an approach to follow what will hopefully lead to a solution. The engineering design 
process is not yet ready to settle for an approach. Instead, the engineering design process 
generates many possible approaches for the solution. With many approaches, an 
engineering analysis and/or a mathematical analysis must be used to decide which 
approach would yield the optimal result. After a rigorous analysis involving all the 
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previous steps in the design proeess, an approaeh is seleeted. The teehnology design 
proeess does not need to perform this analysis beeause there is only one approaeh from 
whieh to seleet. 

As identified by Hailey et al (2005), the end goals of the design proeesses are different. 
The engineering design proeess is more eoneemed with developing and speeifying the 
optimal design than the aetual ereation of it. This eontrasts the teehnology design proeess 
that is more focused on the “hands-on” aspect of building the design than the 
optimization and specification of the design. 

Purpose of the Investigation 

Colorado State University has recently developed a joint education-engineering degree 
program. Students who complete the program receive a nationally accredited 
Accreditation Board for Engineering and Technology (ABET) engineering degree in 
engineering science and a nationally accredited National Council for Accreditation of 
Teacher Education (NCATE) technology teaching license. Some objectives of this 
program include: improving secondary education by placing highly qualified technology 
education teachers in the classroom; encouraging a more diverse population to study 
engineering at an early age by having engineering-trained teachers in the secondary 
classroom; and creating programs to better prepare secondary students to study science, 
technology, engineering, and mathematics (STEM) disciplines in a college undergraduate 
program (De Miranda, Troxell, Siller, & Iverson, 2008). 

Graduates of the Colorado State University education-engineering model develop an 
engineering design philosophy throughout their engineering and K-12 classroom teaching 
experience. One aspect of this philosophy involves using the engineering design process 
to solve problems. When the students enter the secondary classroom, it is likely they will 
teach students to solve problems in the same manner. 

Prior to the education-engineering model at Colorado State University, technology 
teachers were licensed through traditional technology and education studies. During this 
time, there was less emphasis on using the engineering design process to solve problems. 
The pre-service teachers would be taught less mathematical problem solving without 
consideration of optimization in their design. One can assume that a student from this 
program would be likely to teach with less mathematical rigor and less likely to use the 
engineering design process. 

Problem 

This project was focused on what differences exist between traditionally-trained 
technology teachers and engineering-trained technology teachers. In particular, any 
differences between the teaching methods that each group is using in the classroom. We 
also would like to know how the two groups differ in their knowledge and performance 
when preparing to teach in the technology classroom. To perform an analysis of content 
and delivery methods, we need to be able to understand and compare the design lessons 
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for traditionally-trained and engineering-trained teehnology teachers. The overall 
research question for this project was the following: How does traditionally-trained 
technology instruction differ from engineering-trained technology instruction? 

Method 

To better understand the differences between traditionally-trained technology teachers 
and engineering-trained technology teachers, the content and delivery methods must be 
examined. The most efficient way to accomplish this task is by collecting artifacts that 
demonstrate typical teaching lessons from both groups. A common task of the technology 
teacher is to develop lesson plans for class projects. These lesson plans are commonly 
referred to as “design briefs” or “design problems.” Through a national appeal, design 
briefs from traditionally trained practicing technology teachers were acquired. For 
comparison, similar design briefs were collected from technology pre-service teachers 
who completed Colorado State University’s education-engineering program. Due to the 
fact that this program is still in its infancy, there were only three design briefs to 
compare. Therefore, caution should be used when interpreting the data due to the small 
amount of available data. 

A rubric was created around the eleven-step, engineering design process as defined by 
Fide et al. Each component of the process was detailed with four levels of adequacy. The 
top score of a 3 for a component of the process demonstrated complete integration of that 
component into the design process. On the other hand, a score of 0 indicated either a lack 
of use of that component or inadequate integration into the design process. The 
engineering design scoring rubric is shown in Appendix A. The two groups of the design 
briefs were scored according to the defined levels on the rubric. Each design brief was 
scored four times by four different reviewers to lessen the bias of the reviewer. To further 
lessen possible bias, the reviewers were unaware of which design briefs were created by 
engineering-trained teachers and which by technology-trained teachers. 

Findings 

An independent samples t-test between engineering-trained and technology-trained 
teacher-generated design briefs was performed for each of the ten engineering design 
steps. Table 2 displays the means and standard deviations for each step of engineering 
design process. 
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Table 2. 


Means and Standard Deviations for Steps of the Engineering Design Process by Teacher 
Training 


Engineering Trained Technology Trained 


Engineering Design Step 

M 

N 

SD 

M 

N 

SD 

Identify the Need 

2.25 

12 

0.87 

2.17 

12 

1.03 

Define the Problem 

2.33 

12 

0.78 

1.83 

12 

1.03 

Search for Solutions 

1.83 

12 

1.03 

1.42 

12 

1.00 

Identify Constraints 

2.25 

12 

0.87 

1.42 

12 

1.00 

Specify Evaluation Criteria 

2.58 

12 

0.67 

1.08 

12 

0.67 

Generate Alternate Solutions 

1.58 

12 

1.08 

1.75 

12 

0.87 

Engineering Analysis 

2.17 

12 

1.03 

0.17 

12 

0.39 

Decision 

2.00 

12 

1.04 

0.75 

12 

0.45 

Design Specifications 

2.21 

12 

0.94 

0.08 

12 

0.29 

Communication 

2.25 

12 

0.97 

0.50 

12 

0.52 


Once again, due to the small sample size, caution should be used in interpreting the 
results. When evaluated at the alpha level of 0.01, the tests produced statistically 
significant differences for the engineering design steps of Specify Evaluation Criteria, 
Engineering Analysis, Decision, Design Specifications, and Communication. Table 3 
presents a summary of the independent t-test results. Eor each of the statistically 
significant outcomes, the engineering-trained teachers produced higher results. This 
indicates that these five steps of the engineering design process are utilized more by 
engineering-trained teachers than by traditional technology-trained teachers. 


Table 3. 


Independent Samples t-test for Utilization of the Engineering Design Process by Teacher 
Training 


Engineering Design Step 

Mdiff 

T 

Df 

P 

Identify the Need 

0.08 

0.22 

22 

0.83 

Define the Problem 

0.50 

1.34 

22 

0.19 

Search for Solutions 

0.42 

1.01 

22 

0.33 

Identify Constraints 

0.83 

2.19 

22 

0.40 

Specify Evaluation Criteria 

1.50 

5.50** 

22 

<0.001 

Generate Alternate Solutions 

0.17 

0.42 

22 

0.68 

Engineering Analysis 

2.00 

6.29** 

14.08a 

<0.001 

Decision 

1.25 

3.80** 

22 

0.001 

Design Specifications 

2.13 

7.48** 

13.06a 

<0.001 

Communication 

1.75 

5.52** 

22 

<0.001 


aThe t and df were adjusted because of unequal variances 
** Statistically significant at the a = 0.01 level 
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Conclusions/Recommendations 


It should be noted that this is a very preliminary investigation with limited information. 
Since institutions such as Colorado State University produce more engineering-trained 
technology teachers, the analysis should be repeated to check for consistency of results. 
However, the findings from this investigation indicate that engineering-trained teachers 
are more likely to incorporate engineering design into the secondary technology 
classroom. With technology education moving toward infusing more engineering design, 
these results indicate that the training of technology teachers should be examined 
carefully to align with the desired outcomes to teach and become proficient in 
engineering design. This preliminary investigation points to the importance of providing 
technology teachers with engineering science content in order to support them in infuse 
the engineering design process into technology education classrooms. 
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Appendix A 

Engineering Design Brief Scoring Rubric 

Design Brief Reviewer’s Initials 



3 

2 

1 

0 

SCORE 

Identify the 
need 

Objectives are 
clearly 
identified. 
Acknowledged 
who will benefit 
and how. 

Some objectives 
are listed. 
Acknowledged 
who will benefit 
and how. 

Only one 
objective is listed 
or beneficiary is 
not identified. 

No need is 
identified 


Define the 
problem 

Broad definition 
of the problem 
with multiple 
solutions 
possible. 
Objectives are 
clearly defined. 
User 

requirements are 
established. 
Functions are 
established. 

Somewhat 
defined problem 
with a couple of 
possible 
solutions. 
Objectives are 
listed. 

Fiighly defined 
definition of the 
problem with 
only one possible 
solution. 
Objectives are 
vague. 

No definition of 
the problem is 
given. No 
objectives are 
stated. 


Search for 
solutions 

Many different 
pathways are 
researched. Less 
efficient paths 
are eliminated. 
Use of past 
problems or 
experiences to 
solve this type of 
problem. Use of 
creativity for 
new ways to 
solve the 
problem. Search 
findings are 
recorded for 
future reference. 

A few different 
pathways are 
researched. Less 
efficient paths 
are eliminated. 
No use of past 
problems or 
experiences to 
solve this type of 
problem or no 
use of creativity 
for new ways to 
solve the 
problem. 

At least one 
alternate 
pathway is 
researched. No 
use of past 
problems or 
experiences to 
solve this type of 
problem or no 
use of creativity 
for new ways to 
solve the 
problem. 

Only one 
solution. No 
opportunities 
for searching 
for alternate 
solutions. 


Identify 

constraints 

All possible 
physical and 
practical 
limitations are 
identified. 
Limitations are 
not overly 
restrictive to 
innovation. 
Examples 
include cost, 
codes, 

accommodations 
and rules. 

Some physical 
and practical 
limitations are 
identified. 
Limitations 
restrict some 
innovation. 

At least one 
limitation is 
identified. 
Innovation is 
highly restricted 
by the 
limitations. 

No constraints 
are identified. 
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3 

2 

1 

0 

SCORE 

Specify 

Many desirable 

Some desirable 

At least one 

No evaluation 


evaluation 

characteristics of 

characteristics of 

desirable 

criteria are 


criteria 

the solution are 

the solution are 

characteristics of 

specified. 



known. Easy to 

known. 

the solution is 




qualitatively 

Sufficient to 

known. Alternate 




judge alternate 

qualitatively 

solutions are 




solutions based 

judge alternate 

based on at least 




on criteria. 

solutions. 

one criterion. 




Students 

Students 

Examples 




determine 

determine 

include cost. 




importance of 

importance of 

reliability. 




each criterion. 

each criterion. 

weight. 




Four to six 

Two to four 

appearance, ease 




primary criteria 

primary criteria 

of use. 




are listed. 

are listed. 

effectiveness. 




Examples 

Examples 

feasibility. 




include cost, 

include cost. 





reliability, 

reliability. 





weight. 

weight. 





appearance, ease 

appearance, ease 





of use. 

of use. 





effectiveness. 

effectiveness. 





feasibility. 

feasibility. 




Generate 

Many Detailed 

Some alternative 

At least one 

Only one 


alternate 

alternative 

solutions or 

alternative 

solution is 


solutions 

solutions or 

designs to the 

solution or 

generated. 



designs to the 

problem are 

design to the 




problem are 

developed. 

problem is 




developed. Fligh 

Some use of 

developed. 




use of creativity 

creativity for 

Limited use of 




for new 

new solutions. 

creativity for 




solutions. Use of 

Use of checkoff 

new solutions. 




checkoff lists or 

lists or 

No use of 




brainstorming. 

brainstorming. 

checkoff lists or 




Determination of 

Determination of 

brainstorming. 




most valid 

most valid 





approach. 

approach. 




Engineering 

Alternative 

Alternative 

Alternative 

Alternative 


analysis and 

designs are 

designs are 

designs are 

designs do not 


optimization 

compared and 

compared and 

compared and 

exist or are not 



evaluated. Each 

evaluated. Each 

evaluated. Each 

compared and 



design is looked 

design is looked 

design is looked 

evaluated 



at objectively. 

at objectively. 

at objectively. 

mathematically 



Heavy use of 

Some use of 

Little use of 

and through 



math and 

mathematical 

mathematical 

engineering 



engineering 

and engineering 

and engineering 

principles. 



principles to 

principles to 

principles to 




determine 

determine 

determine 




performance of 

performance of 

performance of 




each design. 

each design. 

each design. No 




Mathematical 

Mathematical 

Mathematical 




models/graphs 

models/graphs 

models/graphs 




are generated to 

are generated to 

are generated to 




compare results. 

compare results. 

compare results. 




Results are 

Results are 
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judged for 
reasonability. 

judged for 
reasonability. 





3 

2 

1 

0 

SCORE 

Decision 

Final design is 
carefully chosen 
based on trade- 
offs of each 
solution. 
Engineering 
analysis of 
alternate 
solutions is used 
to determine 
optimal solution. 
Priorities of 
evaluation 
criteria are 
considered 
before a decision 
is made. 

Final design is 
chosen based on 
trade-offs of 
each solution. 
Engineering 
analysis of 
alternate 
solutions is not 
used to 
determine 
optimal solution 
or priorities of 
evaluation 
criteria are not 
considered 
before a decision 
is made. 

Final design is 
chosen. 
Engineering 
analysis of 
alternate 
solutions is not 
used to 
determine 
optimal solution 
or priorities of 
evaluation 
criteria are not 
considered 
before a decision 
is made. 

No decision is 
made or the 
decision is 
made without 
considering 
previous steps 
in the design 
process. 


Design 

specifications 

Details of the 
chosen design 
are described in 
great detail. 
Drawings of the 
design are 
developed. Sizes, 
shapes, notes, 
standards, 
specifications, 
and a bill of 
materials for the 
project are 
stated. Others 
could easily 
replicate the 
design through 
the design 
specifications. 

Details of the 
chosen design 
are described in 
some detail. 
Drawings of the 
design are 
developed or 
sizes, shapes, 
notes, standards, 
specifications, 
and a bill of 
materials for the 
project are 
stated. With 
effort, others 
could replicate 
the design 
through the 
design 

specifications. 

Details of the 
chosen design 
are described. 
Either drawings 
of the design are 
developed or a 
bill of materials 
for the project 
are stated. 

Others could not 
replicate the 
design through 
the design 
specifications. 

Little to no 
documentation 
of the design is 
developed. 


Communication 

Conveying 
information and 
ideas for the 
chosen design. A 
very detailed 
written report 
and oral 
presentation are 
generated to 
communicate to 
other interested 
parties. 

Conveying some 
information and 
ideas for the 
chosen design. A 
written report 
and oral 
presentation are 
generated to 
communicate to 
other interested 
parties. 

Conveying little 
information and 
ideas for the 
chosen design. A 
written report or 
oral presentation 
is generated to 
communicate to 
other interested 
parties. 

No 

communication 
of the design to 
others. 
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STRATEGIES FOR INTEGRATING STEM CONTENT: A PILOT CASE STUDY 


Fred Figliano 
Virginia Tech 


Abstract 

Science, technology, engineering, and mathematics (STEM) literacy is a critical 
component of 2E* century education (AAAS, 1989, 1993; NCTM, 2000; ITEA, 2000). 
America’s current educational reform agenda is spurred by an urgent need for a more 
STEM literate population. The central tenet of STEM literacy is the preparation of people 
who are knowledgeable of the connections between the content and practices of the 
STEM fields. When conceived as an integrated curriculum model designed around 
teamwork and problem-solving environments, STEM education is the ideal pathway for 
achieving such literacy. 

A goal of STEM education is developing interdisciplinary thinkers. Interdisciplinary 
instruction is the act of consciously applying methodology and language from more than 
one discipline to make connections in content that cuts across subject areas (Pring, 1973, 
Jacobs, 1989, Drake, S., & Burns, R. 2004). Pring (1973), in his analysis of the term 
‘integration’ concluded that "the very notion of 'integration' incorporates the idea of unity 
between forms of knowledge and their respective disciplines.” Eor developing 
interdisciplinary thinkers, integrative instruction is the requisite approach. 

The research presented in this document was designed to identify and describe successful 
teaching strategies revealed by studying an exemplar of integrative instruction. The 
results provide a clearer, deeper understanding of the instructional practices found 
effective in delivering integrative instruction and promoting more STEM literate 
students. 

Research Questions 

This study was guided by the following research question: 

How are teaching/learning strategies, as reflected in exemplars of integrative 
instruction, used to promote the purposeful integration of STEM content? 

The data needed to answer this question were gathered through the following set of sub- 
questions. 

1 . What strategies are used in lesson plans to integrate purposefully STEM content? 

2. What instructional strategies do teachers use, based on self-reflection, to 
purposefully teach integrative STEM content? 
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3. What collaborative teacher strategies are used in planning for purposeful 
integration of STEM eontent? 

4. What strategies are used to evaluate student reeognition of multidisciplinary 
connections? 

Research Design 

The design of this research calls for the analysis of exemplar programs featuring 
integrative instruetion. To aeeomplish this analysis, a pilot ease study design was 
implemented. The speeifie type of ease study used was a “single ease: embedded” design. 
The single ease eomes from the use of only one participating site. It is embedded beeause 
data were colleeted from three different instructor perspectives: Biology, History, and 
English. The four researeh sub-questions were individually applied to eaeh of these 
perspeetives and then analyzed as a whole through a proeess of eonvergenee. 

Selection of Subjects 

The types of partieipants needed for this study were those exemplar praetitioners 
practicing integrative instruetional strategies. The review of relevant literature provided 
many sites as being exemplar models of integrated instruction. Meeting Standards 
Through Integrated Curriculum (Drake and Burns, 2004) was the primary souree used 
for these identilieations. Onee these sites had been identified, the list was narrowed down 
to only those in Virginia, this being done because the researeh institution was in Virginia. 
Erom that final list, a selection was made based on the relative proximity to the researeh 
institution. 

Data Collection and Analysis 

To address research question 1, “What strategies are used in lesson plans to integrate 
purposefully STEM eontent?” lesson plans were eolleeted and analyzed from the three 
subject area teaehers (Biology, History, and English). The method of analysis to 
determine if lesson plans indeed refleeted integrative teaehing/learning strategies was a 
theme analysis. 

To address research question 2, “What instructional strategies do teachers use, based on 
self-refleetion, to teaeh purposefully integrated STEM eontent?” data were eolleeted 
through the use of a 14 item open-ended questionnaire. The questionnaire was designed 
to better understand what instructional strategies were used by practitioners to create 
eollaboratively lessons using integrative instruction. 

To address researeh question 3, “What eollaborative teaeher strategies are used in 
planning for purposeful integration of STEM content?”, data were eolleeted through the 
reeording of one planning session that oecurs every six weeks. Dual tape reeorders were 
used to insure the voiees where properly reeorded. The reeording was then transeribed in 
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preparation for coding. A theme analysis was conducted on the transcription to identify 
common threads across the three disciplines. 

To address research question 4, “What strategies are used to evaluate student recognition 
of multidisciplinary connections?”, data were collected in the form of student artifacts. 
Students’ final interdisciplinary projects for several 6 week units were collected with help 
from the instructors. A theme analysis was conducted on this data source to determine 
whether students were making interdisciplinary connections. 

Convergence of Data Sets 

Following the analysis of the four sets of data, they were examined further to identify 
points of convergence. These points of convergence allowed for the creation of a three- 
dimensional picture of integrated instruction. 

Conclusions 

The 15 strategies, as revealed through analysis of data collected in this study, are 
guidelines for instructors wanting to design and implement integrative STEM units. To 
organize these strategies, a sum of frequencies was calculated. These frequencies related 
to how often a particular theme was identified during the coding process which 
specifically related to how often a theme was discussed by participants. 

A careful review of these 15 strategies found there were several broad strategy 
categories: Planning, Implementation, and Evaluation. These strategy categories were 
gleaned from the topics discussed and terms used within each strategy statement. In this 
way, each strategy could be shown to relate to specific aspects of the overall integrative 
instruction process. Organizing strategies in this manner allowed for a logical sequence 
that could be followed by practitioners. 

Six strategies were found to address Planning (see Table 1). Prom the review of these six 
strategies, eight concepts emerge: standards, themes, connections, group work, time, 
trust, support, and brainstorming. 
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Table 1 


Strategies for Planning Integrative Units 



Planning Strategies 

/ 

1 

Lessons are driven by state eontent standards and are based around primary 
and seeondary themes that are agreed upon by all instructors involved, and 
allow for connections to be made among other disciplines. 

64 

2 

Lessons are designed to allow student to work in cooperative groups where 
they produce some product. 

33 

3 

Instructors must arrange for time to meet outside of school to develop 
integrated units. 

32 

7 

All the instructors involved in the integrated unit must trust each other 
enough to give up control of the planning and implementation of the unit. 

12 

9 

During the planning of an integrated unit, it is helpful, though not 
necessary, for faculty to gain the support of school administration. 

9 

11 

Planning for integrated units occurs through brainstorming which is driven 
by state content standard requirements, teacher interest, and teacher 
experiences. 

6 


Implementation is another key eategory, with strategies identified in all four data 
sourees (see Table 2). On elose review of these four strategies, four eoneepts emerge: 
intentional eonneetions, methods of teaehing, attention to eonneetions, keeping traek of 
oonneetions. 

Table 2 


Strategies for Implementation of Integrative Units 



Implementation Strategies 

/ 

4 

As a team, teachers intentionally make connections in content across 
disciplines. 

30 

8 

Methods of teaching can include lecture / discussion, Internet research, 
practice activities, and group work. 

9 

14 

Instructors specifically call attention to connections between disciplines. 

2 

15 

Instructors encourage students to use methods such as graphic organizers to 
keep track of connections. 

2 
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Five strategies were found to address Evaluation, the third eategory (see Table 3). 
From the review of these five strategies, five eoneepts emerged; holistie grading, 
alternative assessment, unit assessment, formatively assess students, and formatively 
assess units. 

Table 3 


Strategies for Evaluation of Integrative Units 



Evaluation Strategies 

/ 

5 

Presentations are watched and graded by the entire instmctor team and a 
single holistic score is awarded. This score counts in each participating 
course. 

29 

6 

Students are assessed for their knowledge of multidisciplinary connections 
through project presentation and class discussion. 

20 

10 

Based on student recognition of multidisciplinary connections units should 
be continually assessed and constantly revised. 

8 

12 

Student research on topics are formatively assessed to identify cross 
disciplinary connections. 

5 

13 

Teachers constantly should be meeting with teachers in other disciplines to 
be sure to make appropriate connections to other disciplines. 

4 


The 15 strategies re fleet a complex iterative process that will take a significant amount of 
time, perhaps years, to fully implement. Practitioners who might use this list as a 
framework for initiating integrative units of instruction in STEM education must 
recognize the complexity and the amount of time needed for implementation. These 
complexities and time requirements were echoed by participants, who, at a collaborative 
planning meeting, stated that if they were asked to give advice to a new teacher who 
wanted to use integrative techniques to teach a unit, they would advise them to start 
small. These 15 strategies, as organized in categories revealed through this study, provide 
a mechanism for faculty to start small. 
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HIGH SCHOOL STEM EDUCATORS’ SELF-EFFICACY BELIEFS AT 

VARIOUS CAREER STAGES 

Brent Holt 
Virginia Tech 


Abstract 

The concept of self-efficacy originated from Bandura’s social cognitive theory, which 
stated that individuals could control their behaviors and thoughts and promote change by 
their own efforts (Bandura, 1982). Bandura (1997) defined self-efficacy as, “the beliefs in 
one’s capabilities to organize and execute the courses of action required to produce given 
attainments.” This definition refers to the concept that individuals who believe they can 
control an outcome or situation through their actions and abilities will tend to be more 
persistent and place more effort in achieving the outcome. Individuals who do not believe 
they can control a desired outcome or situation will tend not to be focused or motivated to 
implement strategies to achieve the outcome. Teacher self-efficacy has been found to 
influence several student outcomes such as achievement (Armor et ah, 1976; Ashton & 
Webb, 1986; Moore & Esselman, 1992; Ross, 1992), motivation (Midgley, Feldlaufer, & 
Eccles, 1989), and students’ own sense of efficacy (Anderson, Greene, & Eoewen, 1988). 
Teachers who have a higher sense of self-efficacy produce increases in student 
achievement when working with struggling students for longer periods of time and create 
learning environments that are more responsive to students (Woolfolk, Rosoff, & Hoy, 
1990). Ashton and Webb, (1986) found that teachers with higher self-efficacy were more 
inclined to maintain positive learning environments that were student focused. Teachers 
with low self-efficacy tended to have more rigid and controlling classrooms (Woolfolk & 
Hoy, 1990) and ignored or criticized students who answered questions incorrectly 
(Gibson & Dembo, 1984). 

Variations in Teachers’ Sense of Self-Efficacy Across Time 

Even with a broad research base in self-efficacy, little has been done to study the 
development of teacher self-efficacy and whether these beliefs remain stable over time. 
Tschannen-Moran, Hoy, & Hoy (1998) found that very little evidence exists as to how 
self-efficacy beliefs solidify across a teaching career. It appears to be unclear as to 
whether teacher self-efficacy changes or stabilizes across a career. Brown & Gibson 
(1982) found that teachers at later stages in their careers had lower sense of self-efficacy, 
while another study found that outstanding teachers had no differences in their self- 
efficacy across career stages (Pigge & Marso, 1993). While it might be reasonable to 
assume that teacher self-efficacy will change over time, predicting the nature of the 
change, one has to consider the two components of teacher self-efficacy, personal self- 
efficacy and teaching self-efficacy. 

Changing Self-Efficacy Beliefs 

Teacher self-efficacy is a very important construct related to numerous behaviors found 
in teachers as well as students. Even though a higher sense of self-efficacy produces 
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higher measures of student achievement (Ashton & Webb, 1986), teacher beliefs appear 
to be quite stable and resistant to change (Brousseau, Book, & Byers, 1988). Making a 
transition to a new conceptual understanding or the application of a new pedagogical 
approach is difficult because preexisting beliefs are tenacious even in the face of 
contradictory evidence (Kagan, 1992). Bandura (1997) cautioned that positive changes in 
self-efficacy only come through “compelling feedback that forcefully disrupts the 
preexisting disbelief in one’s capabilities” (p. 82). Teacher self-efficacy has been shown 
to have the potential to change over time. DeMoulin (1993) noted a fluctuation in 
teaching self-efficacy levels from pre-service and novice teachers to experienced teachers 
presented with similar tasks. As teachers become more experienced, modifying self- 
efficacy beliefs becomes more difficult. Even if teachers are exposed to new workshops 
or presented new teaching methods, there is a resistance to this change. Guskey (1988) 
confirmed that change is gradual and difficult after intervention, and programs requiring 
change need to be accompanied by encouragement, support, and feedback to be effective 
(Stein & Wang, 1988). 

Method 

Quantitative and qualitative research methods were employed in the study of STEM 
teachers’ sense of self-efficacy. An online survey was administered to selected STEM 
teachers in Virginia to obtain demographic data and data relating to their levels of self- 
efficacy and factors associated with self-efficacy. Erom the results of this survey, a 
selected sample of participants was interviewed with the goal of increasing an 
understanding of self-efficacy beliefs. 

Participants 

Participants were selected from among the population of Virginia secondary STEM 
educators. Specifically, participants (teachers) were recruited from the various high 
school programs across the State of Virginia, including urban as well as rural school 
districts. Eists of superintendents’ and high school principals’ e-mail addresses were 
compiled from Virginia high school Web pages. Superintendents were contacted through 
a personal email requesting permission to survey teachers under their jurisdiction. After 
obtaining those permissions, the researcher asked school principals via email to forward 
an email note provided to them by the researcher to each of the STEM teachers in their 
school. In accordance with the policies of Virginia Tech’s Institutional Review Board, 
this email asked STEM teachers to participate in the study, and directed them to the 
online survey instrument. 

Eight participants were purposefully identified for follow-up interviews based upon their 
teaching self-efficacy scores as measured in the quantitative component of this study. 
More specifically, they were selected randomly from four “pools” of STEM teachers 
identified with high and low self-efficacy in rural and urban schools. 
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Instrumentation Design and Development 

Data were gathered using an online survey condueted via the Internet. The survey 
eonsisted of two seetions. The first section asked participants to provide demographic 
information, including gender, age, education, educational major, county in which 
present school is located, type of community (rural or urban) in which the school is 
located, years of teaching experience, years of teaching in a STEM program, enrollment 
level of existing school, and type of community in which the teacher grew up. If the 
teacher had five or fewer years’ experience, they were also asked: utilization of a mentor, 
perceived support of mentor, perception of first year teaching experience, perception of 
student teaching experience, and perceived quality of teacher preparation. 

The second section of the survey consisted of 24 questions using the Teachers’ Sense of 
Teacher Efficacy Scale (TES) (Tschannen-Moran& Woolfolk Hoy, 2001). This 
instrument measured teacher self-efficacy based on 24 items obtained from Bandura’s 
(1997) efficacy scale. Each item was measured using a 9-point Eikert-type scale. 

Quantitative Research Design 

The quantitative approach used in this study is descriptive in nature in that variables were 
not manipulated and that the sole purpose of this study was to gain more information 
about STEM teachers’ self-efficacy. Analysis was done on a number of factors to 
determine the degree of correlation between the factors and teacher self-efficacy. 

Qualitative Research Design 

To acquire more in-depth information of teachers’ perceptions, open-ended interviews 
were conducted with STEM teachers whose self-efficacy values fell outside of the norm 
for their demographic or experience level. The rationale for the use of a qualitative 
approach was to conduct research in the teachers’ natural setting attempting to interpret 
phenomena in terms of the meanings that teachers could articulate. It was hoped that this 
approach could add further evidence regarding factors that affect teacher self-efficacy 
over teachers’ careers. A qualitative, phenomological methodology was chosen because it 
seemed well suited for encouraging teachers to tell their stories, to reflect on and describe 
their human experiences, and explore their perceptions within their STEM field. These 
data were elicited through personal interviews and were tape recorded. 

Research Questions 

1 . Do self-efficacy measures for classroom management, instructional strategies, and 
student engagement differ among novice and experienced STEM teachers across the 
STEM disciplines? 

2. Which factors contribute significantly to variations in novice and experienced STEM 
teacher self-efficacy across the STEM disciplines? 


28 



3. Does STEM teacher self-efficacy show a point of stabilization and does this point 
vary across the STEM disciplines? 
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INVESTIGATING MIDDLE-SCHOOL TEACHERS’ 
ENGINEERING SUBJECT MATTER AND 
PEDAGOGICAL CONTENT KNOWLEDGE 


Morgan Hynes 
Tufts University 

Abstract 

Including engineering edueation in the K-12 setting is a relatively reeent initiative. 
Engineering is a rieh field of study that allows students to see the value of what they are 
learning, apply interdiseiplinary knowledge to eontexts that make sense to them, and to 
ereate and explore the world around them. As industry, organizations, and sehool systems 
transition into ineluding engineering edueation into the K-12 elassroom, edueational 
researeh in the subjeet area of engineering will require greater attention. The researeh 
proposed here grew out of the need to understand the nature of the task teaehers faee and 
what they need to know when teaehing engineering. The proposed dissertation study aims 
to examine middle-sehool math, seienee, and eomputer teaehers’ use and development of 
subjeet matter and pedagogieal eontent knowledge as they teaeh an engineering unit 
foeused on the engineering design proeess (see Appendix A). The proposed qualitative 
study involves observing and interviewing a small (-'5-8) eohort of teaehers as they 
implement the same engineering unit in the Boston Publie Sehools. 

Introduction 

The researeh proposed here grew out of the need to understand the nature of the task 
teaehers faee and what they need to know when teaehing engineering. The eentral 
researeh questions eentered on this need and foeus on the knowledge that teaehers use 
and develop as they teaeh an engineering unit. 

Research Questions 

1 . What subjeet matter knowledge do middle sehool math and seienee teaehers use 
and develop as they teaeh an engineering unit foeusing on the engineering design 
proeess? 

2. What engineering pedagogieal eontent knowledge do middle sehool math and 
seienee teaehers know, use, and develop as they teaeh the said engineering unit? 

3. How do math and seienee teaehers eonneet their subjeet matter and pedagogieal 
eontent knowledge the same and differently when teaehing the said engineering 
unit? 

Three eonstruets are at the eore of these questions — subject matter knowledge, 
pedagogical content knowledge, and expertise. The first two eonstruets eontribute to the 
knowledge base of teaehing. Subjeet matter knowledge and pedagogieal eontent 
knowledge are both eontent-speeifie, meaning that, for eaeh subjeet (e.g., engineering). 
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they may contain different knowledge. The third construct, expertise, considers how 
teachers develop towards expertise in their engineering subject matter and pedagogical 
content knowledge. 

Subject matter knowledge (SMK) entails the mix of math, science, and engineering 
content, concepts, principles, process skills, and fundamentals along with knowledge of 
engineering design — ^which usually involves the application of the aforementioned 
principles. Previous research in other subjects (e.g., math) has shown that a teacher’s 
depth of SMK impacts their ability to teach (Ball & McDiarmid, 1990; Ma, 1999) as well 
as their teacher self-efficacy (Hoy & Davis, 2006) or belief in their ability to impact 
change with their students. 

Pedagogical content knowledge (PCK) can be thought of as a teacher’s knowledge of 
how to teach a specific subject (Shulman, 1986). PCK includes knowledge of students 
(i.e., their misconceptions and understandings), teaching strategies (i.e., real-world 
examples, probing questions), and lesson management. Previous research has illustrated 
the importance of teachers knowing and understanding their students backgrounds and 
conceptions (Driel, Verloop, & Vos, 1998; Magnusson, Krajcik, & Borko, 1999; 

Peterson, 1988; Veal, Tippins, & Bell, 1998). While PCK is commonly thought of as 
knowledge teachers develop through their experience and practice (Veal, Tippins, & Bell, 
1998), others have shown that PCK can be developed with focused study outside the 
classroom (Carpenter, Fennema, & Franke, 1996). The proposed research hopes to 
capture the development of teachers’ engineering PCK. 

The construct of expertise considers what makes someone an expert in a particular 
domain. In terms of teaching, expertise considers what knowledge, skills, and strategies 
teachers develop to be more effective as a teacher (Berliner, 1986, 1994). Since the goal 
is to capture what PCK teachers use and develop, it will be important to consider what 
knowledge, skills, and strategies are developed in the pursuit of engineering teaching 
expertise. 

Methodology 

Six grade 5-8 teachers will be selected to participate in this study and will all be teaching 
the same LEGO robotics engineering curriculum (see Appendix B) developed by the 
researcher and collaborators. Each of the teachers will have previously participated in a 
summer teacher professional development workshop led by the researcher or 
collaborators. Data from these teachers will be collected in the form of: (1) semi- 
structured interviews, (2) videotaped classroom observations, (3) hands-on think-aloud 
tasks, and (4) student projects. Each of the six teachers will be interviewed and observed 
a minimum of four times over the course of the study. 

Within specified interviews, the teachers will be presented with hands-on, think-aloud 
tasks that will aim to elicit both their subject matter knowledge and pedagogical content 
knowledge related to engineering. These methods of data collection are intended to allow 
the teachers to both verbalize what they are doing and experiencing, and allow the 


32 



researcher to then confirm what the teachers were doing in the classroom (Appleton, 
2003; Dawkins, Dickerson, & Butler, 2003; Veal, Tippins, & Bell, 1998). Veal, Tippins, 
and Bell (1998) highlighted the success of using vignettes or teaching scenarios for 
teachers to work through during interviews. The hands-on think- aloud tasks will include 
an example LEGO artifact along with the teaching scenario to best represent what a 
teacher using the LEGO robotics curriculum would encounter. These tasks were 
developed out of the need to identify the subject matter knowledge the teachers were 
using as they approached teaching situations, which was seen as a shortcoming in a prior 
pilot study (Hynes, 2007). Einally, the students’ final projects will be assessed using two 
methods. Eirst, the teacher will assess their students’ projects to reveal how well the 
teacher can identify the SMK addressed in the design as well as the PCK they may have 
used to assist the students in getting to their final project. Then the researcher will assess 
the student projects from each class to serve as a cross-case comparison of the teachers. 
These two methods will highlight both the teacher’s ability to recognize the SMK 
incorporated into the projects and the success the teacher had in teaching their students 
(PCK) to implement the subject matter into their designs. 

Miles and Huberman’s (1994) qualitative data analysis approach will be applied in the 
analysis of the interview, task, observation, and student project data. The approach 
incorporates different types of data into displays and matrices to help reduce and organize 
data for analysis. The data is then analyzed by noting patterns and themes, clustering 
data, making comparisons, and noting relationships and then organizing the data into 
conceptually ordered matrices and charts, which help tell the story. A complete content 
analysis of the curriculum and results from the previous pilot study (see Hynes, 2007) 
provided the basis for the coding scheme that has been developed to this point (see 
Appendix C). Both within-case analysis for each teacher and cross-case analysis among 
the teachers will be used to examine the data. 

Implications 

The results from this study may help inform engineering educators prepare teachers, 
develop teacher resources, and create curriculum that will foster students’ knowledge and 
interest in engineering. The research may also provide valuable insight into methods of 
analyzing teacher knowledge and how it can be researched further. If nothing else a small 
handful of teachers and their students will experience the excitement of engineering with 
LEGO! 
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Appendix A - Engineering design process models that act as the basis of the middle- 
school engineering curriculum. 


[Graphic Unavailable 
for Reproduction] 



(Massachusetts DOE, 2001) 


(Newfoundland and Labrador DOE, 2007) 
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Appendix B - Robotics curriculum outline 


Challenge 
(estimated time) 

Spatula design 
(1-2 hours) 


Wheelchair design 
(1 hour) 


Design selection 
(1 hour) 


Design selection 
continued 
(1 hour) 

Design selection 
continued 
(1 hour) 

Programming 
(1-2 hours) 


Programming with 
sensors (1-2 
hours) 

Assistive device 
final project (4-8 
hours) 


Description 


Students will build a spatula out of LEGO that must be long and strong. The students 
will test the spatulas as a group, discuss design strategies that made them strong, then 
redesign the spatulas, and retest them. The teacher and students will then discuss the 
Engineering Design Process (EDP) and how they worked through the process in their 
challenge. 

Students will build a LEGO wheelchair that must be able to hold a bottle of water, be at 
least 8 inches tall, be able to roll freely, and survive a drop test. The teacher will begin 
the challenge with a discussion about constraints and design criteria and their role in the 
EDP. 

This is a multi-part challenge. The discussion at the beginning and throughout the class 
revolves around selection and design criteria and how they play a role in the EDP and 
product development. First, the students have to select the “besf’ LEGO wheelchair 
drive train based on a number of criteria. 

The discussion for the second part of the challenge revolves around using orthographic 
drawings, learning how gears work, and constructing the model. The students use a set of 
orthographic drawings to construct the model they chose. Each model incorporates gears, 
and the teacher presents information on how gears work as they complete the challenge. 
Finally, the students, with the constructed models, test and evaluate the models. They 
find out how fast they go, how well they travel up an inclined ramp, how well they turn, 
and how well they traverse terrain. The teacher then has the students discuss and 
compare all the different models that were tested. 

The programming challenge begins with the teacher introducing the students to the 
concept of computer programming and, specifically, the ROBOLAB programming 
language. After this brief introduction, the students use the models they built in the 
previous challenge to complete simple programming challenges (e.g., program the 
wheelchair to go forward for 2 seconds and stop). The challenges progress in difficulty. 
The teacher begins this challenge discussing how sensors work. The teacher relates 
sensors to both the human senses and to the LEGO sensors. Students are then given 
programming challenges that incorporate the LEGO light and touch sensors (e.g., 
program the wheelchair to go forward until it crosses a black line). 

The final design challenge asks the students to create an assistive device using the LEGO 
toolset with a minimum of 1 sensor, 1 motor, and the RCX brick. The students research a 
need or problem that they will propose to design a solution for. They then determine the 
criteria they will address and use the EDP to create, test, redesign, and finally market 
their solution to the class. 
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Appendix C - SMK and PCK coding schemes 


Subject Matter Knowledge Codes 


Math 

Concept 

Description 

Ml 

Graphing 

Using data and plots to estimate and optimize 

M2 

Measurement 

Measuring physical dimensions, speed 

M3 

Estimation 

Estimating measurements, weights, speeds, etc 

M4 

Symmetry 

Identifying symmetry in design (balance and stability) 

M5 

Statistics 

Statistics as selection and reporting criteria 

M6 

Percentage 

Percentages as selection and reporting criteria 

M7 

Ratios 

Calculating and using ratios in gears 

M8 

Scale 

Identify and use scale in engineering drawings 

M9 

Other 





Science 


SI 

Static forces 

Gravity, load, structural forces 

S2 

Torque 

Rotational force about a point (force x distance) 

S3 

Moment of Inertia 

"Angular mass", resistance to angular motion 

S4 

Trusses 

Transfer of forces 

S5 

Bending moment 

Resistance object has to bending 

S6 

Friction 

Resistance to motion 

S7 

Speed 

Rate at which an object moves 

S8 

Simple machines 

Screw, gear, wheel and axle, wedge, lever 

S9 

Kinetic forces 

Forces in motion, kinetic frictional forces 

SIO 

Other 





Engineering 


El 

Eng. Design 

Eight steps of the engineering design process 

E2 

Testing 

Evaluating design features and constraints using systematic tests 

E3 

Tradeoffs 

Design features and constraints and their pros and cons 

E4 

Constraints 

Criteria or limitations that apply to the design of an artifact 

E5 

Prototypes 

Functional model of a product to test certain features 

E6 

Eng. Drawings 

Orthographic, isometric, views 

E7 

Specifications 

Set of requirements to be satisfied 

E8 

Comm. System 

Source, encoder, transmitter, decoder, storage, retriever, receiver 

E9 

Comp. Programming 

If/then, while, loops, iterations, etc. 

ElO 

Control 

Using sensors/feedback to control a robot or device 

Ell 

Electronics 


E12 

Other 





LEGO/ROBOLAB 


LI 

Connections 

Flow to connect different LEGO pieces together 

L2 

Stability 

Sturdy and stable construction 

L3 

Gears 

Flow to use the various types of LEGO gears (connections, applications, 
orientations) 

L4 

RCX 

Flow to operate and connect to the RCX 

L5 

Motors 

Flow to operate and connect to the motors 

L6 

Sensors 

Function, operation, and connection of sensors 

L7 

Programming 

Fundamentals of programming with ROBOLAB 
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Pedagogical Content Knowledge Codes 


Real World Examples 


RWl 

Physical in-class example 

RW2 

Talk about example 

RW3 

Demonstration/video example 

RW4 

Other 




Strategy for student understanding 


SUl 

Question/Inquiry 

SU2 

Demo/Physical 

SU3 

Instruction/Direction/Suggestion 

SU4 

Other 




Knowing students 


KSl 

Student misconception 

KS2 

Student background education 

KS3 

Student background/culture 

KS4 

Did not know student background (education or culture) 

KS5 

Other 
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EXAMINATION OF ENGINEERING DESIGN IN CURRICULUM CONTENT 
AND ASSESSMENT PRACTICES OF SECONDARY TECHNOLOGY 

EDUCATION 

Todd R. Kelley 
University of Georgia 


Abstract 

This descriptive study examined the current status of technology education teacher 
practices with respect to engineering design. Participants were drawn from the current 
International Technology Education Association (ITEA) membership database. A survey 
instrument gathered data about the extent to which engineering design concepts are 
incorporated into the curriculum content and assessment practices employed by 
secondary technology educators. Moreover, the survey identified challenges faced by 
technology educators when seeking to implement engineering design. Current curriculum 
content that addresses engineering design concepts consisted of the following seven 
subsets: (a) engineering design, (b) engineering analysis, (c) application of engineering 
design, (d) engineering communication, (e) design thinking, (f) engineering and human 
values, and (g) engineering science. The instrument was developed from current research 
in technology education (Asunda & Hill, 2007; Rhodes & Childress, 2006; Smith, 2006; 
Gattie & Wicklein, 2007). 

Introduction 

The field of technology education has a history of experiencing curriculum reforms that 
generate new program titles with little curriculum changes (Akmal, Oaks, & Barker, 
2002; Clark, 1989; Sanders, 2001). Considering the history of resistance to change in the 
field of technology education, questions arise about the current curriculum shift to move 
to engineering design as a content focus. 

Recently, there have been new curricula designed to infuse engineering content into 
technology education such as Project ProBase, Principles of Engineering', Project Lead 
the Way, Principles of Technology, Engineering Technology, ax\A Introduction to 
Engineering (Bearing & Daugherty, 2004.) Certainly, research was needed to determine 
to what degree engineering design content was being presented in technology education. 
Moreover, a need to understand where technology educators are in practice regarding an 
engineering design focus was expressed by leaders in technology education (NCETE 
meeting report, Oct, 2006). It was clear that a descriptive study could better identify the 
depth of implementation of engineering design content infused into technology 
education. 

Purpose of the Study 

This descriptive study examined the degree to which technology educators are 
implementing elements of engineering design in their curricula. A full sample was taken 
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of all secondary technology educators who were members of the International 
Technology Education Association (ITEA) as of September 2007. The sample consisted 
of all high school technology teachers regardless of whether they indicated they taught 
engineering design in their classrooms. The survey instrument gathered data about the 
degree to which engineering design concepts were incorporated into the curricula content, 
assessment practices employed by secondary technology educators, and challenges to 
implementing engineering design concepts in the secondary technology education 
curriculum. 

Research Questions 

1. To what degree does the current curriculum content of secondary technology 
education programs reflect engineering design concepts? 

2. To what degree do current assessment practices of secondary technology educators 
reflect engineering design concepts? 

3. What selected challenges are identified by secondary technology educators in 
teaching engineering design? 

Methodology 

This descriptive study examined the degree to which technology educators were 
implementing elements of engineering design in their curriculum. This descriptive study 
sought to describe the current engineering design content and assessment practices using 
the results of four recent research studies (Asunda & Hill, 2007; Childress & Rhodes, 
2008; Smith, 2006; Gattie & Wicklein, 2007) to create items for the survey instrument. 

Instrumentation 

The first section of the survey instrument gathered data about degree to which 
engineering design concepts were incorporated into technology education curriculum 
content. The curriculum content items were created from the results of Childress and 
Rhodes study (2008) and Smith’s study (2006) to create the framework for defining 
engineering design curriculum content in seven categories, see Table 1. Participants were 
required to respond to each curriculum content item twice, ^ox frequency of use and for 
time per typical use, using a six-point Eikert scale, see Table 2. 

The second section of the survey instrument consisted of identified assessment practices 
for evaluating engineering design activities (Asunda & Hill, 2007). Participants rated 
their level of agreement regarding their assessment practices of engineering design 
activities with the identified assessment practices presented in the instrument. 

A third section of the survey instrument identified teacher challenges relative to 
implementing curriculum changes to infuse engineering design into technology education 
curricula. Participants used a five-point Eikert scale to rate their levels of experience 
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identifying teacher challenges. The five-point Likert scale was as follows: Never = 0, 
Rarely = 1, Sometimes = 2, Very Often = 3, and Always = 4 (Lodico, Spaulding, & 
Volegtle, 2006). One question was open-ended allowing participants to identify any other 
challenges faced. The final section of the survey instrument collected general 
demographic information of each participant. 

Results 

Using the Krejcie and Morgan (1970) method for calculating sample size for a given 
population size, the appropriate sample size for this study was determined to be 285. The 
final results of the study yielded a total of 226 respondents, therefore, the results of this 
study cannot be generalized to the entire population. 

The results of this research yielded a 0.982 Cronbach’s Alpha for internal consistency. 
Engineering Communication was the highest ranked category with a group mean score of 
2.80. In the Engineering Design category was the highest-ranking individual item 
(measured by time per typical use) use of computer-aided design to construct technical 
drawings with a mean score of 3.35. Also, the item use technical drawings to construct 
or implement an object, structure, or process (mean score of 3.30), was high ranking. 

The emphasis of CAD in technology has been discovered in other status studies (Dearing 
& Daugherty, 2004; Sanders, 200; Warner & Mumford, 2004). 

Another result of particular interest is that the second highest ranked item measured by 
time per typical use was develop basic student skills in the use of tools with a mean of 
3.32. It appears that the field of technology education has not moved far from its 
industrial arts roots. As a matter of fact, a similar survey item, developing skill in using 
tools and machines, was the highest ranked item in the SflAP project (1979) and Schmitt 
and Felly study (1963) according to Sanders (2001). 

The lowest ranking categories based on composite scores for total instructional time 
were. Engineering and Human Values (6.21 hours for traditional schedule; 6.06 hours for 
block schedule). Engineering Science (7.06 hours for traditional schedule; 8.88 hours for 
block schedule), and Engineering Analysis (14.41 hours for traditional schedule; 14.16 
hours for block schedule); see Figure 1 and Figure 2. These results reveal that there is 
less emphasis on the use of math to predict design results and a low emphasis on 
optimization techniques. Some might question if engineering design is being properly 
taught when these are key engineering design elements (Hailey, Erekson, Becker, & 
Thompson, 2005; Hill, 2006; Gattie & Wicklein, 2007). Similar results were obtained in 
the assessment practice and teacher challenges section. See appendix for those results. 

Conclusion 

Current research reveals that technology education teachers believe there are potential 
benefits of an engineering design focused curriculum (Gattie &Wicklein, 2007). 

However, those benefits may never be realized unless our field is properly informed as to 
the status of its practitioners regarding the implementation of engineering design into the 
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technology education classroom; this study sought to provide such information. The 
evidence from this study reveals that technology education curriculum content currently 
emphasizes career and technical education skills such as CAD and general tool skills 
providing rationale for the field to develop an engineering career pathway for high school 
students. This would end the argument that dates back to Dewey and Prosser of 
vocational education versus general education and would provide a clear mission for the 
field as suggested by some (Wicklein, 2006). 
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Appendix 


Table 1 

The Seven Categories of Engineering Design Content 

Seven Categories of Engineering Design Content 

Engineering Design 

Engineering Analysis 
Application of Engineering Design 
Engineering Communication 
Design Thinking as It Relates to Engineering Design 
Engineering and Human Values 
Engineering Science 


Table 2 

Teaching Style Scale Conversion 


How Often? (Erequency) 

How Many Minutes? (Time) 

Eikert 

Wording 

Traditional 

(meets 5 days a 
week) 

Block 

Wording 

Traditional 

(50 minutes per 
period) 

Block 

(90 minutes per 
period) 

0 

Never 

0 

0 

None 

0 min. 

0 min. 

1 

A few times 
a year 

5 days 

5 days 

A few 

minutes per 
period 

5 min. 

9 min. 

2 

1 or 2 times 
a month 

14 days 
(1.5*9.1) 

7 days 
(1.5*4.6) 

Less than 
half the 
period 

15 min. 

30 min. 

3 

1 or 2 times 
a week 

55 days 
(1.5*36.8) 

28 days 
(1.5*18.4) 

About half 

25 min. 

45 min. 

4 

Nearly 

everyday 

129 days 
(3.5*36.8) 

64 days 
(3.5*18.4) 

More than 
half 

37.5 min. 

67.5 min. 

5 

Daily 

1 84 days 

92 days 

Almost all 
period 

50 min. 

90 min. 


Assumptions: Traditional schedule meets 5 days a week, 50 minute period, 184 day school year. Typical 
A/B and 4x4 block scheduling meets for 92 days for 90 minutes. 
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Table 3 

Engineering Design 


Engineering Design Content 

Mean 

Frequency 

SD 

Frequency 

Mean 

Time 

SD 

Time 

understand engineering design is an iterative 
proeess 

3.03 

1.21 

2.27 

1.20 

understand ereativity is an important 
eharacteristie for engineers to apply in design 

3.33 

1.21 

2.51 

1.34 

reeognize that there are many approaehes to 
design and not just one design proeess 

3.26 

1.32 

2.42 

1.28 

recognize engineering as a potential career 
option 

3.05 

1.31 

2.12 

1.22 

are able to identify good and bad design 

2.96 

1.19 

2.40 

1.16 

believe in his/her ability to design a solution 
to a technological problem 

3.27 

1.19 

2.58 

1.31 

Average Group Mean / Std Dev 

3.15 

1.24 

2.38 

1.25 


Table 4 

Engineering Analysis 





Engineering Analysis Content 

Mean 

Frequency 

SD 

Frequency 

Mean 

Time 

SD 

Time 

understand that knowledge of science and 
mathematics is critical to engineering 

3.44 

1.20 

2.61 

1.25 

apply engineering science principles when 
designing solutions 

3.15 

1.25 

2.59 

1.29 

use measuring equipment to gather data for 
troubleshooting, experimentation, and 
analysis 

3.09 

1.25 

2.69 

1.26 

use physical and/or mathematical models to 
estimate the probability of events 

2.12 

1.42 

1.93 

1.35 

use optimization techniques to determine 
optimum solutions to problems 

2.09 

1.41 

1.82 

1.38 

use models or simulations to study processes 

2.82 

1.40 

2.58 

1.40 

Average Group Mean / Std Dev 

2.79 

1.32 

2.37 

1.32 
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Table 5 

Application of Engineering Design 



Mean 

SD 

Mean 

SD 

Application of Engineering Design Content 

Erequency 

Erequency 

Time 

Time 

apply knowledge for manufacturing products to 
the engineering design 

2.62 

1.22 

2.39 

1.28 

identify problems that could be solved through 
engineering design 

2.82 

1.23 

2.48 

1.24 

understand no perfect design solution exists 

2.91 

1.41 

2.24 

1.31 

conduct reverse engineering to analyze product 
design 

2.02 

1.34 

2.26 

1.51 

organize and manage design process for optimal 
use of materials, processes, time, and expertise 

2.50 

1.33 

2.39 

1.34 

design, produce, and test prototypes 

apply research to designing products, processes. 

2.89 

1.34 

3.15 

1.39 

and materials 

2.65 

1.24 

2.62 

1.32 

develop skills to use, manage, and assess 
technology 

2.94 

1.29 

2.65 

1.31 

demonstrate the ability to handle open-ended/ 
ill-defined problems 

2.79 

1.30 

2.50 

1.33 

develop basic students' skills in the use of tools 

3.46 

1.26 

3.32 

1.34 

understand design often requires tradeoffs 

2.86 

1.24 

2.44 

1.25 

Average Group Mean / Std Dev 

2.77 

1.29 

2.59 

1.33 

Table 6 

Engineering Communication 


Mean 

SD 

Mean 

SD 

Engineering Communication Content 

Erequency 

Erequency 

Time 

Time 

communicate design ideas orally, through 
presentations, and graphics 

2.96 

1.35 

2.94 

1.29 

communicate through writing technical reports 
use technical drawings to construct or 

2.03 

1.29 

2.25 

1.39 

implement an object , structure, or process 

3.34 

1.26 

3.30 

1.25 

visualize in three dimensions 

3.26 

1.31 

3.19 

1.32 

develop and maintain an engineering design 
portfolio 

use computer-aided design to construct 

2.54 

1.87 

2.07 

1.71 

technical drawings 

3.39 

1.52 

3.35 

1.49 

apply the rules of dimensioning 

3.09 

1.49 

2.98 

1.51 

apply rules of manufacturing tolerance 
use basic computer applications such as word 
processors, spreadsheets, and presentation 

2.10 

1.35 

2.00 

1.37 

software 

3.27 

1.39 

3.15 

1.36 

Average Group Mean / Std Dev 

2.89 

1.42 

2.80 

1.41 
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Table 7 

Design Thinking Related to Engineering Design 


Design Thinking Related to Engineering 

Mean 

SD 

Mean 

SD 

Design Content 

Erequency 

Erequency 

Time 

Time 

think eritically 

3.65 

1.10 

3.15 

1.22 

synthesizes simple parts into complex 
systems 

Apply systems thinking - understanding and 

2.73 

1.25 

2.61 

1.29 

considering the multiple facets of a design 
solution result in positive and negative 
impacts 

apply brainstorming and innovative concept 

2.58 

1.42 

2.34 

1.34 

generation 

have the ability to approach open-ended/ ill 

3.24 

1.20 

2.98 

1.30 

defined problems 

2.80 

1.41 

2.62 

1.44 

Average Group Mean / Std Dev 

3.00 

1.28 

2.74 

1.32 

Table 8 

Engineering and Human Values 


Mean 

SD 

Mean 

SD 

Engineering and Human Values Content 

Erequency 

Erequency 

Time 

Time 

understand how engineers put ethics into 
practice 

1.75 

1.23 

1.76 

1.32 

are aware of social, economical, and 
environmental impacts on design solutions 

2.31 

1.24 

2.21 

1.24 

understand that the solution to one problem 
may create other problems 

2.47 

1.28 

2.23 

1.30 

consider cost, safety, appearance, and 
consequences of design failures 
take human values and limitations into 

2.47 

1.34 

2.25 

1.33 

account when designing and solving 
problems 

2.27 

1.33 

2.07 

1.31 

apply knowledge of basic ergonomics to 
engineering design process 

2.04 

1.32 

1.95 

1.35 

Average Group Mean / Std Dev 

2.22 

1.29 

2.08 

1.31 
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Table 9 

Engineering Science 



Mean 

SD 

Mean 

SD 

Engineering Scienee Content Frequeney 

Frequency 

Time 

Time 

apply math and seienee to the engineering 
design proeess 

apply knowledge of basie meehanies to the 

3.15 

1.26 

2.84 

1.24 

engineering proeess 

apply knowledge of basie statics and 

strengths of materials to engineering design 

2.88 

1.33 

2.69 

1.29 

process 

2.02 

1.28 

1.98 

1.32 

apply knowledge of dynamics to the 

engineering design process 

use of algebra to solve problems or predict 

1.81 

1.40 

1.76 

1.39 

results to design solutions 

use geometry to solve problems or predict 

2.19 

1.47 

1.98 

1.35 

results to design solutions 

use trigonometry to solve problems or predict 

2.60 

1.35 

2.30 

1.32 

results to design solutions 

1.65 

1.37 

1.58 

1.34 

apply knowledge of material process to 
engineering design process 

2.37 

1.35 

2.19 

1.37 

Average Group Mean / Std Dev 

2.33 

1.35 

2.16 

1.33 

Table 10 





Engineering Design Category Rankings for Erequency of Use 




Total Group Mean 

Total Group SD 

Rank Engineering Design Content Category 


Frequency 

Frequency 

1 Engineering Design 


3.15 


1.24 

2 Design Thinking Related to Eng. Design 


3.00 


1.28 

3 Engineering Communication 


2.89 


1.42 

4 Engineering Analysis 


2.79 


1.32 

5 Application of Engineering Design 


2.77 


1.29 

6 Engineering Science 


2.33 


1.35 

7 Engineering and Human Values 


2.22 


1.29 
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Table 1 1 

Engineering Design Category Rankings for Time Per Typical of Use 


Rank 

Engineering Design Content Category 

Total Group Mean 
Time 

Total Group SD 
Time 

1 

Engineering Communication 

2.80 

1.41 

2 

Design Thinking Related to Eng. Design 

2.74 

1.32 

3 

Application of Engineering Design 

2.59 

1.33 

4 

Engineering Design 

2.38 

1.25 

5 

Engineering Analysis 

2.37 

1.32 

6 

Engineering Science 

2.16 

1.33 

7 

Engineering and Human Values 

2.08 

1.31 


Table 12 

Assessment Practices for Engineering Design Projects 




Assessment Practices 

Mean 

Erequency 

SD 

Erequency 

Mean 

Time 

SD 

Time 

use support evidence / external research 
(research notes, illustrations, etc) 

2.32 

1.38 

2.25 

1.37 

provide evidence of formulating design 
criteria and constraints prior to designing 
solutions 

2.33 

1.45 

2.19 

1.43 

use design criteria such as budget, 
constraints, criteria, safety, and functionality 

2.45 

1.34 

2.31 

1.39 

provide evidence of idea generation strategies 
(e.g. brainstorming, teamwork, etc.) 

2.92 

1.46 

2.69 

1.50 

properly record design information in an 
engineer's notebook 

2.01 

1.76 

1.78 

1.64 

use mathematical models to optimize, 
describe, and/or predict results 

1.72 

1.43 

1.62 

1.39 

develop a prototype model of the final design 
solution 

2.69 

1.43 

2.87 

1.55 

work on a design team worked as a functional 
inter-disciplinary unit 

2.53 

1.50 

2.79 

1.60 

Average Group Mean / Std Dev 

2.37 

1.47 

2.31 

1.48 
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Traditional Schedule Breakdown of Total Hours 


6.21 


Key : 

□ Engineering and 
Value 



19.58 


□ Engineering 

□ Engineering 

□ Application 
Engineering 

■ Engineerin 
Communication 

O Design Thinking 
to Engineering 


■ Engineering Design 


Figure 1. Composite Score for Traditional Schedule. 


Block Schedule Breakdown of Total Hours 


6.06 


Key : 

□ Engineering and 
Value 



□ Engineering 

□ Engineering 

□ Application 
Engineering 

■ Engineerin 
Communication 

□ Design Thinking 
to Engineering 


■ Engineering Design 


Figure 2. Composite Score for Block Schedule . 
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Total Hours Per Assessment Strategy 


Key :Assessment Strategy 


□ use support evidence / 
external research (research 
notes, illustrations, etc) 


8.15 


14.46 


6.92 


18.33 



9.76 


18.00 


■ provide evidence of 
formulating design criteria 
and constraints prior to 
designing solutions 

□ use design criteria such as 
budget, constraints, criteria, 
safety, and functionality 

□ provide evidence of idea 
generation strategies (e.g. 
brainstorming, teamwork, 
etc.) 

■ properly record design 
information in an engineer's 
notebook 

□ use mathematical models to 
optimize, describe, and/or 
predict results 

□ develop a prototype model of 
the final design solution 


□ work on a design team 
worked as a functional inter- 
disciplinary unit 


Figures. Composite Score for Assessment Strategies for Traditional Schedule. 
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Total Hours Per Assessment Strategy 


Key :Assessment Strategy 
□ use support evidence / 
external research (research 
notes, illustrations, etc) 


7 C.ri 



18.5 


■ provide evidence of 
formulating design criteria 
and constraints prior to 
designing solutions 

□ use design criteria such as 
budget, constraints, criteria, 
safety, and functionality 

□ provide evidence of idea 
generation strategies (e.g. 
brainstorming, teamwork, 
etc.) 

■ properly record design 
information in an engineer's 
notebook 

□ use mathematical models to 
optimize, describe, and/or 
predict results 


□ develop a prototype model of 
the final design solution 


□ work on a design team 
worked as a functional inter- 
disciplinary unit 


Figure 4. Composite Score for Assessment Strategies for Block Schedule. 
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COMPARING QUESTION POSING CAPABILITY ACROSS 
HIGH SCHOOL CURRICULA: A RESEARCH PROPOSAL 

Benjamin Franske 
University of Minnesota 


Rationale 

Our lives and world seem to be teeming with problems; how to save on gasoline; how to 
keep food from spoiling; how to maintain our ability to feed a growing population; how 
to proteet endangered speeies. We live in a world whieh seems defined by problems. For 
all of recorded history, we, as a society, have relied on science and technology to help 
address problems such as these. From the earliest methods for farming to the invention of 
the wheel, sail and ship and, more recently, the advances brought about by electronics, 
computers and the Internet, we see the importance of technology as a method for 
addressing a growing number of problems. Still, all of these advances in science and 
technology would remain purely intellectual curiosities were it not for people who can 
see a practical value to the knowledge and who have an ability to apply it to solve 
problems. As with most useful and productive services, over time, a profession of people 
who use science and technology to solve real world problems has emerged. Many of 
those involved in this type of problem solving are identified by themselves or others as 
engineers. 

The U.S. Department of Labor describes an engineer as someone who applies the 
principles of science and mathematics to solve technical problems economically (2007). 
Thus, when we talk about solving real world problems using technology and scientific 
discoveries in a systematic way, we are really talking about engineering problem solving. 
It naturally follows, then, that engineers who are most familiar and most successful with 
problem solving and have the required skills will develop the highest quality solutions to 
the problems they examine. As we do not live in an ideal world, it is not enough for 
engineers to design the best possible solution to problems. Even in the definition 
provided by the Department of Labor, we can see the influence of commercialization and 
economics. The solutions devised by engineers must not only be technically and 
scientifically feasible, but must also be economically feasible. This wrinkle in 
engineering problem solving which requires that a solution be economical can add 
significant complexity to the problem solving process and necessitates that the best and 
most desirable engineers have superior problem solving skills. 

Although the success of an engineer is largely determined by their ability to solve 
complex and ill-structured “real world” problems, engineering schools often do a poor 
job of teaching students how to deal with these types of problems (Jonassen, Strobel & 
Lee, 2006). The solving of problems with unclear goals and unlisted constraints is not 
something that students are able to learn through the structured problem solving to which 
they are most commonly introduced in engineering classes (Jonassen et ah, 2006). The 
problems discussed in school are typically designed to have a correct, or at least 
apparently optimal, solution to which the students’ solutions can be compared. In the real 
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world, problems often have more than one appropriate solution and ean require the 
balanee of oonfUeting goals and other eomplex issues. 

More recent research has suggested that even the word or story problems that engineering 
students might be exposed to do not appear to provide adequate transfer to the solving of 
complex, ill-structured problems (Jonassen et ah, 2006). It is not just engineers who 
benefit from the practical application of science and technology to solve problems. As 
our daily lives have evolved and become more complex, we have exposed ourselves to an 
increasing number of technological problems. For example, the majority of Americans 
living today take for granted the ability to control the climate in their homes through 
heating and cooling systems. The function of a building has evolved from a simple 
shelter from the elements to something expected to keep us comfortable year around, but 
the increasing cost of energy is slowly causing us to re-think our living habits. Thus, the 
homeowner must now balance the desire for a comfortable home with the reality of the 
costs for maintaining such a home. These seemingly contradictory goals might be 
achievable, or the situation improved, through the application of scientific and 
technological principles in a problem solving process, and the better that process, the 
more satisfactory the end result. 

Problems 

It may not seem necessary to define what is meant by a problem. After all, we frequently 
use the term and nearly everyone has experienced a problem of some kind, but when 
embarking upon a study of problems, it soon becomes evident that “problem” is not a 
universal term and can have multiple meanings which must be differentiated. 

Psychologist Karl Duncker (1945) begins his work on problem solving by describing 
problems as what occur when someone has a goal but does not yet know how to meet that 
goal. This description of a problem as something which blocks the move from an existing 
state to a desired state is echoed by Simon and Newell (1972). Along these lines, it is 
possible to describe what blocks this transition as a difficulty. The Oxford English 
Dictionary describes the most common uses of problem to be “a difficult or demanding 
situation; a matter or situation regarded as unwelcome, harmful or wrong and needing to 
be overcome; a difficulty” (2008) a description which clearly fits those provided by 
Duncker, Simon and Newell. 

It is not merely enough though to describe a problem as a difficulty. Those problems 
faced by engineers and designers, while they may be difficult, are more than a difficulty. 
Merriam-Webster’s Dictionary defines a problem as “a question raised for inquiry, 
consideration, or solution” (2008) which is a more appropriate definition in this case. 
Getzels (1982) suggests that in these cases, instead of being a difficulty, developing the 
problem is itself the primary goal and what remains then is execution. Furthermore, it 
becomes possible to classify problems based on whether the problem exists or is created, 
whether the problem is suggested by the solver or another and whether a known solution 
exists or must be devised. This method for classification of problems led Getzels (1964; 
1982) to describe ten common types of problems, revolving around whether the problem 
exists or is to be discovered, or whether there are known steps to solutions. 
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Jonassen (2000) takes a slightly different taet and suggests two eritical attributes for an 
issue to be a problem. First, there must be a situation with an unknown deseribed as a 
diserepaney between a eurrent state and goal state. This is similar to the dehnition used 
by Simon and Newell (1972). Seeondly, there must be some soeial, cultural or intellectual 
value to hnding or solving the unknown. The value could be either intrinsic or extrinsic, 
but the key component is that someone feels that it is worth finding the unknown. 

Jonassen (2000; 2006) believes that there are thee essential types of problems which form 
the basis for problem classification. Puzzle problems are characterized by having a single 
correct solution which reached using a specific procedure (Jonassen, 1997). Although 
multiple methods may accomplish the same end result, only the single most efficient 
method is deemed to be the correct one. These are problems which have most commonly 
been associated with the study of cognitive problem solving, for example, by Simon and 
Newell (1972), and include the Tower of Hanoi, water jug problems and the nine dot 
problem. Jonassen (1997) suggests that, while these are interesting from an initial 
research perspective, they map poorly onto complex real-world problems and, as such, 
are not relevant to school learning or everyday practice. 

Well-defined or well-structured problems are those to which people are most conditioned 
to solve and are most familiar, especially in the school setting. For example, when a math 
or science teacher writes problems for an exam or assigns “homework problems”, they 
are most frequently well-defined problems. This is likely because well-defined problems 
have a definite solution process which requires the application of concepts, rules and 
principles from a given knowledge domain (Jonassen, 1997). In other words, well- 
structured problems are good for checking basic understanding and facts which is often 
the desired outcome of homework and exams. Jonassen (1997; 2000) describes well- 
structured problems as having a well-known initial state, a defined goal and known 
method for arriving at a solution. Although not explicitly identified as well-structured, the 
first and second type of problems identified by Getzels (1964; 1982) are really examples 
of well-defined problems. These are given problems with standard methods for solving 
the problem, and solutions can be compared to determine correctness. 

One common misconception about well-defined problems which Jonassen (Jonassen, 
1997) wishes to dispatch is the idea that skills learned in solving them will easily transfer 
to real world, ill-structured problems. The third problem category is the ill-structured or 
ill-defined problem. Unlike puzzle problems and well-structured problems, this category 
of problem is frequently tied to a specific context, and the information required to solve 
the problem is not all available in the problem statement (Jonassen, 1997; Jonassen, 

2000). Many, if not most, of the problems encountered in daily life are of the ill- 
structured variety (Jonassen, 1997). For example, problems found or given to engineers 
and designers do not often contain all of the information required to solve the problem in 
the problem statement. There is also significant room for creativity in these types of 
problems as more than one correct solution may exist. The problem will often require the 
application of multiple domains of knowledge, and judgment calls must be made by the 
problem solver (Jonassen, 1997). The problem types Getzels (1964; 1982) identifies as 
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three through ten eould be deseribed as ill-structured problems in that multiple solutions 
might exist, and the problem and/or the method for solving it is not fully understood. 

For the purposes of this study, the problems of interest are ill-defined questions raised for 
inquiry, consideration or solution. This is undoubtedly the most frequently encountered 
type of problem in the real world and that which is faced by designers and engineers on a 
daily basis. Although designers and engineers are often asked to “fix a problem” or are 
given what initially appear to be problems, they are often more accurately described as 
dilemmas or issues. There is usually no immediately apparent path to solution, and 
devising a solution will require the application of critical thinking skills and creativity. 
The dilemma or issue must also be somehow turned into a problem which can be solved. 

Problem Posing 

The study of problem solving and the desire to utilize it in schools and workplaces is not 
new. Educational researcher John Dewey (1910) proposed a five-step problem solving 
process which included: (1) felt difficulty, (2) problem clarification, (3) identification of 
possible solutions, (4) testing of solutions, and (5) verification of results. The four step 
heuristic model for problem solving proposed by mathematics researcher George Polya 
(1957) is similar to that of Dewey as it includes: (1) understanding the problem, (2) 
devising a plan, (3) carrying out the plan, and (4) looking back on all of which can be 
seen in Dewey’s five steps. One problem with models such as these is that they make the 
complex task of problem solving appear to be deceptively simple and can hide some of 
the processes which result in the most desirable solutions. In his seminal study of 
problem solving, psychologist Karl Duncker (1945) suggests that the process of finding a 
solution is more accurately seen as the continual reformulation of the problem. Over 
time, this problem reformulation leads to the discovery of “essential” properties of 
the solution which, given knowledge of the domain, will, in turn, dictate an appropriate 
solution to the problem. 

It may initially seem like a radical suggestion that the formulation of the problem is more 
essential than the solution, but Duncker is not alone in this belief. Scientists, Albert 
Einstein and Eeopold Infeld, noted the importance of problem formulation in their 
discussion of the evolution of physics: The formulation of a problem is often more 
essential than its solution which may be merely a matter of mathematical or experimental 
skill. To raise new questions, new possibilities, to regard old problems from a new angle, 
requires creative imagination and marks real advance in science (1961, p. 92). 

Einstein illustrates the importance of problem formulation by discussing the problem of 
determining whether light travels instantaneously or whether it occupies time, as sound 
does, a question posed by Galileo. Although the crude instruments of Galileo’s time 
prohibited him from answering this question once he had formulated the question, he was 
able to discern an experimental procedure which could be used leaving the work to be 
done a matter of technical and experimental skill (Einstein & Infeld, 1961). Clearly, the 
experiment itself is not the most difficult part of this problem. Instead, the formulation of 
the problem is the true challenge, and if we are to be good at the solving of problems, we 
must be good at finding problems. 
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Sometimes, when the topic of problem finding comes up, it is asked “why do we need to 
seek out more problems when our world is already full of them?” It is certainly the case 
that our world has many issues, dilemmas and quagmires, but for these there is no 
immediately apparent path to solution (Getzels, 1979). They are not solvable in their 
current form. Returning to our earlier discussion of problems, this can further complicate 
the categorization of problems as the initial dilemma may be presented to the problem 
solver, but the problem to be solved remains to be found. Take for example early prairie 
farmers who were able to harvest an abundance of grain on their remote but fertile 
ground. Someone may consider the problem to be that we did not have enough roads to 
transport grain to markets and ponder how to get roads built in these remote areas. 

Taking this view, the problem domain has already been narrowed to building roads. As it 
turns out, roads are not the only way to transport grain and may not be the most efficient 
for a specific situation. If a river is nearby, or a number of farmers deposit grain in an 
elevator near them to which a railroad could be built, these may be better solutions. 
Solving the problem for these farmers is aided by the finding or formulation of an 
underlying problem, the transportation of grain from farm to market, and the posing of 
questions such as the feasibility of centrally collecting grain prior to shipment or the 
proximity of a navigable river which may provide a better solution. 

If the ability to solve complex and ill-structured problems is a critical skill, and the 
formulation of problems or problem posing a critical and early step in the process of 
solving these types of problems, should we not focus attention upon problem finding as a 
desired outcome of technology and engineering education? Unfortunately, problem 
finding has been largely neglected by researchers, as a whole, (Getzels, 1979) and in the 
field of technology education (Lewis, Petrina, & Hill, 1998). It is the goal of this study to 
identify the state of problem finding in the high school classroom as it relates to the 
solving of technological problems and to compare the problem finding abilities of 
students from a wide variety of backgrounds. 

Problem Solving 

Once a problem has been found, devising a solution to the problem can begin. The most 
basic description of problem solving comes from the seminal work of Newell and Simon 
(1972) which describes problem solving as the transition from an initial situation to a 
goal state by a narrowing of the problem space. The initial state, sometimes referred to as 
the problem state, encompasses the understanding of the situation as it exists. Take, for 
example, the dilemma of energy efficiency given above. In this case, the problem state 
could include the thickness of walls, amount of insulation, price of energy, type of 
heating and cooling system, area climate and other similar factors. The goal state is the 
desired result and embodies the solution to the problem. In our example of home energy 
efficiency, this could be the replacement of the heating system, additional insulation, use 
of passive solar design or other solutions to reduce climate control costs while improving 
efficiency. The link between the problem state and the goal state is the search for 
solutions through the narrowing of the problem space. This narrowing occurs as the 
problem solver searches through the information to which they have access seems 
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relevant to the problem ineluding things in their memory and any researeh they eonduet. 
Eventually, the problem solver narrows the problem spaee enough to determine the 
solution to the problem (Newell & Simon, 1972). 

Even in the example of home energy effieieney, we ean begin to see defieieneies in 
Newell and Simon’s deseription of problem solving. The problem has more than one 
solution, and the problem spaee eannot be systematieally narrowed until a solution 
beeomes obvious. Complex and ill-defined problems sueh as those most eommonly faeed 
by engineers and designers eannot be solved in Newell & Simon’s simple problem spaee 
model (Middleton, 2005). 

Reeognizing the defieieneies in the simple problem spaee model as well as the extensive 
use of design in Australian teehnology elassrooms, Middleton (2005) modified Newell 
and Simon’s simple problem spaee model to aeeount for the eharaeteristies of design 
problems ineluding the ill-struetured nature and potential existenee of multiple solutions. 
The revised model replaees the problem state with a “problem zone” whieh enables 
starting with an ill-defined and eomplex problem about whieh little is known. The goal 
state is replaeed by a “satisfieing zone” indieating an understanding that more than one 
eompeting goal may exist, and a balanee may need to be struek as well as the 
understanding that multiple solutions may exist. Einally, the simple narrowing of the 
problem spaee by searehing is replaeed by a eomplex seareh and eonstruetion proeess 
where numerous proeedures are used whieh may be eonstrueted or emerge. Another 
eritieal aspeet of the revised model is that it is not a linear one. There remains a 
vaeillation between the problem zone and the seareh and eonstruetion spaee as well as 
between the satisfieing zone and the seareh and eonstruetion spaee (Middleton, 2005). 
This vaeillation allows the problem to be redefined and the solution reexamined as the 
problem solving eontinues. 

A Focus on Design and Engineering 

Teehnology edueation in the United States, like industrial arts and manual training before 
it, has traditionally foeused on domain knowledge and produetion skills rather than on 
intelleetual proeesses. In 1983, the landmark government report A Nation at Risk was 
published. Among other indieators that the Ameriean system of publie edueation was on 
a dangerous path, it was reported that high sehool students were laeking in intelleetual 
skills, sueh as problem solving, whieh should be expeeted of them (The National 
Commission on Exeellenee in Edueation, 1983). The eoneems identified in A Nation at 
Risk were reiterated in 1991 with the publieation of What Work Requires of Schools: A 
SCANS Report for America 2000. Again, intelleetual skills and problem solving were 
identified as laeking in Ameriean students (The Seeretary's Commission on Aehieving 
Neeessary Skills, 1991). Taking note of this shift in the workplaee, Johnson (1992) wrote 
in the Journal of Technology Education about the eritieal nature of intelleetual skills and 
suggested that the field adopt a eurrieulum emphasizing intelleetual skills sueh as 
problem solving through ill-struetured, design oriented problems. 
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In 2000, the International Technology Education Association published the Standards for 
Technological Literacy, a set of content standards for the study of technology in schools. 
Out of the twenty standards, four are entirely focused on the design process, and several 
others make note of the importance of design (International Technology Education 
Association, 2000). Since many cognitive scientists consider design to be a special case 
of problem solving (Newell & Simon, 1972), the inclusion of design in standards such as 
these can be interpreted as an explicit inclusion of problem solving. Although technology 
educators in the United States have been slow to include explicitly design in their 
curriculum, other parts of the world have truly embraced design as a foundation for 
technology education. The British, too, were concerned about the gap between education 
and industry, but their wake up call came with the release of the Crowther Report in 1959 
(Gradwell, 1996), several decades before A Nation at Risk and the SCANS report were 
released in the United States. Beginning in 1963, British researcher Gerd Sommerhoff 
opened the Technical Activities Centre at Sevenoakes School in Kent (Gradwell, 1996). 
This center was designed squarely to address the gap between education and the 
workplace, especially the field of engineering, through student designed projects 
requiring creativity and problem solving skills (Gradwell, 1996). 

Erom the late 1960s through the 1970s, the ideas promoted by Sommerhoff were 
propagated to other schools in Great Britain, and design based problem solving slowly 
became a regular part of the curriculum (Gradwell, 1996). The late 1980s and early 1990s 
saw the introduction of a national curriculum in the United Kingdom aimed at ensuring 
that all students learn essential knowledge and skills. One of the subjects selected for 
inclusion in this curriculum was Design & Technology, a subject heavily influenced by 
the ideas of Sommerhoff almost thirty years earlier (Atkinson, 1990; Gradwell, 1996). 
The advent and subsequent revisions of the national curriculum in the United Kingdom 
have firmly entrenched the idea of design and problem solving within their technology 
curriculum while the United States continues to struggle with the idea though some 
progress is being made. 

In recent years the field of technology education in the United States has been moving 
toward a more engineering and design focused curriculum. Articles by respected 
technology education researchers (Eewis, 2005; Wicklein, 2006) as well as the inclusion 
of design standards in the ITEA Standards for Technological Literacy (2000) indicate the 
increasing interest in engineering and design as content for the study of technology. As 
engineering and design curriculum continue to propagate through American schools, 
there will need to be continued research to promote understanding student problem 
solving both in how problem solving can be nurtured in students as well as appropriate 
methods for the assessment of problem solving. 

Problem Statement 

Complex and ill-structured problems such as those commonly found within the design 
and engineering fields require that solvers have the ability to define and question the 
problem itself through a process of problem finding, but the research on problem finding 
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in technology and engineering education has not been extensive, and we do not know 
whether high school engineering curricula support this critical skill in problem solving. 

Guiding Research Questions 

1 . What can we say about the types of problems found by students? 

2. How do students who have taken high school engineering coursework compare 
on measures of problem finding ability and the types of question posed to those 
with traditional technology education coursework and those without any kind of 
engineering or technology coursework? 

3. How do students in rural, exurban, suburban and urban areas compare on 
measures of problem finding ability and the types of questions posed? 
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THE EFFICACY OF CROSS-DISCIPLINE REPRESENTATIONS 
FOR ILL-DEFINED IAS CONCEPTS 

Steven Rigby 
Purdue University 


Abstract 

A universal problem in our soeiety is the dramatie inerease in the number of seeurity 
threats, risks, and vulnerabilities to our nation’s computer systems, data, and 
infrastructure. Our future depends upon the problem solving and thinking abilities of 
professionals entering the Information Assurance and Security (IAS) field. These 
professionals will be faced with many problems that are complex, ill-defined, and 
multidisciplinary in nature. But how do we, as educators, prepare these professionals to 
be successful? Are traditional approaches sufficient for the complexity they will face? 
Providing complex, ill-structured learning activities increases problem solving abilities of 
the learner and increases expertise, however, most of the research that has been done to 
date has been specific to domains of knowledge that could be classified as well-defined 
and not multidisciplinary. Much is not known especially in domain areas that are 
complex, ill-defined, and multidisciplinary such as information assurance where the 
knowledge domain is dynamic, complex, multidisciplinary, and rapidly changing with 
many concepts being ill-defined. The purpose of this study is to investigate how inner 
and cross-discipline representations affect learners’ conceptual understanding of ill- 
defined IAS concepts. More specifically, this study investigates the efficacy of presenting 
cross-discipline representations of ill-defined concepts to learners to improve conceptual 
understanding. 

Introduction 

The demand for professionals able to solve IAS problems is steadily increasing, and 
academia is now faced with the challenge of preparing these professionals for the 
complex, ill-defined problems that await them (PITAC, 2005). While some institutions 
are experimenting with case study approaches to teaching problem solving, others are 
developing laboratory exercises to develop troubleshooting skills. Although instruction 
can take many forms, it is worth reevaluating how we think about IAS instruction and the 
nature of thinking skills. The knowledge required to solve the complex problems of IAS 
involves many well-defined and ill-defined concepts. Well-defined concepts are those 
considered “concrete” and are clearly defined with described features and characteristics 
(Smith & Ragan, 1999). In mathematics terminology, a “well-defined” concept is one in 
which the input will always provide the same answer (Hall, 1959). For example, the 
exclusive or (XOR) logical operation would be considered a well-defined concept 
because performing this operation on the same two operands will always produce the 
same results. Ill-defined concepts are those that are less structured and “abstract” with no 
single clear characteristic and are usually based on situations and the understanding of 
other related concepts (Smith & Ragan, 1999). When discussing ill-defined concepts, the 
discussions usually begin with “it depends”. The attributes and relationships between ill- 
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defined eoneepts are of key importanee for solving “real-life” problems. Aeademia is 
eurrently teaehing well-defined and ill-defined IAS eoneepts to students through 
edueational experienees with the hope that eoneepts learned in sehool will transfer to the 
everyday problems they will face professionally, but are these educational experiences 
successful? 

Research suggests that providing complex, ill-structured learning activities increases 
problem solving abilities of the learner and increases expertise, however, much of the 
research done to date has been specific to domains of knowledge that could be classified 
as well-defined and not multidisciplinary, such as mathematics. There is much that is not 
known especially in domain areas that are complex, ill-defined, and multi-disciplinary. In 
fields such as information assurance, the knowledge domain is dynamic, complex, 
multidisciplinary (see definition of terms), and rapidly changing with many concepts 
being ill-defined. This is not to say that IAS does not include well-defined concepts, 
rather that the well-defined concepts have connecting relationships with ill-defined 
concepts and, oftentimes, both are needed in order to solve problems. 

Of central importance for educators of IAS disciplines is what factors influence transfer 
(see definition of terms) of these well-defined and ill-defined concepts. According to 
Mestre (2002): 

Cognitive research has converged on the conclusion that transfer is better 
if people have learned initially in a way that fosters deep, abstract 
understanding of central principles of a field. . . . One method for achieving 
such deep, abstract understanding is to involve students in multiple 
examples illustrating a central principle drawn from an equivalence class 
where the surface characteristics of problem vary as widely as possible. 

(plO) 

This suggests that cross-discipline representations foster deep learning of abstract 
concepts which enhances transfer and increases expertise (see definition of terms). One 
unexplored area is to what extent inner-discipline representations and cross-discipline 
representations affect the learning of ill-defined concepts such as those found in the field 
of IAS. Up to this point, research has mainly focused on well-defined concepts while the 
problems faced most by IAS professionals will require conceptual understanding of ill- 
defined concepts. This presents a need to research how inner and cross-discipline 
representations affect learners’ mental models or schemata of ill-defined concepts. Of 
particular interest is to what extent cross-discipline representations increase a learner’s 
conceptual understanding of an ill-defined concept. The closer the learner’s conceptual 
understanding is to that of experts, the better the learner will be able to approach new 
problems in a meaningful way. 

Research Question 

The question this study is to examine is whether cross-discipline representations are 
more effective than inner-discipline representations in increasing students’ conceptual 


64 



understanding of complex, ill-defined concepts like those found within the knowledge 
domain of technology and IAS. 


1 . Are representations of ill-defined eoneepts within the same discipline as 
effective as cross-discipline representations in increasing expertise in 
learners? 

2. To what degree do additional cross-discipline representations of ill-defined 
concepts increase the conceptual understanding of learners? 

Methodology 

Previous research suggests that multiple representations are an important factor to 
increase conceptual understanding with cross-discipline representations increasing 
expertise more than inner-discipline representations. It is expected that this will also 
apply to ill-defined concepts. The hypotheses of this study will investigate the effect 
inner and cross-discipline representations have on students’ learning of ill-defined 
concepts. The following hypotheses indicate the expected outcomes of this study. 

1 . Cross-discipline representations are more effective than inner-discipline 
representations in increasing conceptual understanding of an ill-defined 
concept. 

2. Each additional cross-discipline representation increases conceptual 
understanding of an ill-defined concept at a functional rate. 
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COMPLEX SYSTEMS IN ENGINEERING AND TECHNOLOGY EDUCATION: 
A MIXED METHODS STUDY INVESTIGATING THE ROLE SOFTWARE 
SIMULATIONS SERVE IN STUDENT LEARNING 


Douglas J. Walrath 
Utah State University 


This research will determine if students receiving complex systems instruction recognize 
patterns and underlying elements of complex systems more effectively than students 
receiving traditional instruction. Complex systems were investigated with an analytic 
(reductive) approach in a control group and with a synthesis approach in the treatment 
group. Exploration of this top-down approach to learning complex systems counters 
traditional bottom-up methodologies, investigating systems and sub-systems at the 
component level. The hypothesis was that students experiencing complex systems 
scenarios in a computer-based learning environment would outperform their counterparts 
by constructing a greater number of explanations with emergent-like responses. 

A mixed method, experimental, pretest/posttest, control group Triangulation Design 
research study was designed for high school students enrolled in an Introduction to 
Technology and Engineering course. A pretest consisting of one open-ended near transfer 
problem and one far transfer problem was administered, investigating the generation of 
reductive (clockwork) and complex (emergent-like) mental models. Following treatment, 
an analysis of covariance failed to reveal statistically significant evidence supporting the 
hypothesis. However, qualitative data in the form of student transcriptions, daily lab 
reports, and data entry worksheets revealed evidence of emergent-like response and 
behaviors. 

It is important for high school students to be able to synthesize information from 
disparate courses (Frank, 2005; Jacobson & Wilensky, 2006). The American education 
system tends to teach core classes in separate rooms and seldom explains how concepts 
from both disciplines can be used to solve real problems (Thode & Thode, 2002). “For 
nearly a century. Western society in general and American society in particular, has been 
dominated by a form of thinking and an approach to life that is narrowly reductive and 
deeply analytical” (Pink, 2006, p. 2). Frank (2005) explains this analytical focus as, “the 
traditional approach in engineering and technology teaching [which] is bottom-up, i.e. 
component to system” (p.20). However, Pink (2006) states that “what’s in greatest 
demand today isn’t analysis but synthesis - seeing the big picture, crossing boundaries, 
and being able to combine disparate pieces into an arresting new whole” (p. 66). 

An alternative approach capitalizing upon interdisciplinary connections, which may be 
effective in technology education, is that of complex systems teachings (Charles, 2003; 
Jacobson & Wilensky, 2006). Exploration of a top-down approach to learning complex 
systems counters traditional bottom-up methodologies, investigating systems at the 
component level. This approach provides students with a holistic perspective exploring 
scenarios with cultural, environmental, economic, political and societal interactions. 
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Engaging students in complex systems teaching aligns with technology education 
standards four through seven, focusing on technology and society relations. 

Complex systems approaches have not been researched in Engineering and Technology 
Education (EXE) programs. Complex systems thinking has its roots anchored in scientific 
domains, which, due to their compulsory nature, make them well suited for complex 
inquiry. However, EXE programs are particularly well suited to investigate complex 
systems as “technology education teachers are in a unique position in that their 
curriculum is often more flexible, and they have the opportunity to present the ‘big 
picture’ to their students by tying the other areas together in realistic activities” (Thode & 
Thode, 2002, p. 15). 

Therefore, an experimental study was conducted with high school students enrolled in a 
technology education class, utilizing a stratified sampling method for control or treatment 
groups. The study addressed the research question: Can high school students’ exposure to 
complex systems scenarios within a software simulation enhance the generation of 
emergent framework mental models, as demonstrated by the ability to create emergent- 
like explanations as they are applied to near and far transfer problems? 

The site for this study was Mid-Western High School (MWHS). With an enrollment of 
1767 students (2007-08), MWHS is located in a suburban community of a Midwestern 
city of 200,000. Eighteen students enrolled in a freshmen-level Introduction to 
Technology and Engineering class were randomly assigned to groups. 

The dependent measures for this study were complex responses in the form of Emergent 
Eramework Metal Models (EEMM) and reductive responses, in the form of Clockwork 
Mental Models (CWMM). An alternative for EEMM and CWMM was a lack of 
understanding, identified as No Model (NM). The independent variable was the method 
of complex systems instruction. Control group students received reductive instruction 
focusing at the subsystem and component level during the construction of a robot. 
Treatment participants received synthesis instruction within a global warming simulation 
(C02EX) with embedded complexity concepts. 

A pretest and posttest was administered with one open-ended near transfer problem and 
one open-ended far transfer problem coded and analyzed with Charles’ (2003) 
Ontological Mental Model Taxonomy to measure group differences. A concurrent form 
of analysis was selected for this mixed methods Triangulation Design research study. A 
one-way analysis of covariance (ANCOVA) was conducted for each of the three 
dependent variables: Emergent Eramework Mental Models (EEMM), Clockwork Mental 
Models (CWMM) and No Models (NM). A qualitative analysis was completed on the 
three primary forms of data collected: worksheets from the software simulation 
intervention, student’s daily lab reports, and nine sets of transcriptions from teams of 
three students working within the intervention. 

Quantitative and qualitative evidence tend to support one another in this research. 
ANCOVA results do not provide statistically significant evidence of emergent-like 
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understanding following the intervention. Qualitative data reveal a predominance of 
student statements, actions, and behaviors within the simulation displaying reductive 
characteristics. Not surprisingly students relied upon a trial and error problem-solving 
approach typifying strategies frequented in technology education. However, over the 
course of the month-long treatment, students provided examples displaying emergent-like 
characteristics across all three qualitative data forms (see Appendix A). Verbal 
exchanges, between students working in teams of three, tended to indicate that through 
repetitive trial and error approaches students exhibited a novice-level complex systems 
understanding. Revelation of underlying complexity elements was demonstrated in 
spoken and written form, but most often through a pattern of decisions revealed on 
C02FX data entry worksheets (Appendix B). 

Fenner’s research (2000) into emergent phenomena found that students lack the cognitive 
capacity to represent accurately emergent-like concepts: “that is, even though students 
might possess considerable domain knowledge, they do not necessarily possess the ways 
of thinking that can help them analyze phenomena appropriately” (p. 804). It is possible 
treatment participants’ intuition led them to believe certain emergent- like happenings 
were occurring in the C02FX simulation. However, failing to possess the tools to analyze 
independently, their answers subsequently lacked sufficient evidence to be coded as 
EFMM. Therefore, no differences were found at posttest. 

Beyond learning complexity terminology and concepts, the challenge is one of 
demonstrating complex systems understanding over time. Transferring information to a 
new domain with emergent-like component beliefs would serve as a practical 
demonstration. However, that did not occur in this study. This could partially be due to 
the treatment group’s repeated exposure in a single complexity scenario. Gick and 
Holyoak (1983) found that a single analogous event failed to provide adequate abstract 
representation transferring to new settings. However, transfer increased when two or 
more analogs sharing similar characteristics were used. Utilizing several simulations 
embracing similar complexity concepts could provide complementary abstract 
representations facilitating transfer, whereas, a single unit failed to demonstrate statistical 
significance in this research. 

Clearly, the need exists for more research in this area. Complex systems and student 
learning research is still in its infancy within science domains. As technology education 
represents a natural, multidisciplinary learning environment for presenting the “big 
picture” research, this domain would be a logical extension of current efforts in science. 
Recommendations for future research would include studies pertaining to pedagogy, 
curricular materials, and learning tools. Many important questions need to be addressed, 
to include: How does students’ conceptual understanding change over time? What 
strategies are appropriate based on age? What role can longitudinal studies play? 
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Appendix A - Qualitative Data Source Findings 
Table 1 


Qualitative Triangulated Data Sources: Coded Themes Across Three Trials 


Data Forms; 

Trial 1 

Trial 2 

Trial 3 


Singular Causality 

Singular Causality 

Singular Causality 

C02FX 

Student 

Trial and Error 
Tradeoffs 

Trial and Error 
Tradeoffs 

Trial and Error (or) 
Experimentation 

Transcripts 

Big Change = Big 
Effect 



(Themes) 

Change Over Time 




Additive Effects 




Large Input =17.67 

Large Input =12 

Large Input = 12.33 

C02FX 

Student 

Worksheets 

Small Input = 
15.67 

Small Input = 
18.33 

Small Input = 
26.33 

(Mean Responses) 

No Change = 5 

No Change = 8 

No Change = 10.33 


Singular Causality 

Singular Causality 

Singular Causality 

Daily Lab 
Reports 

(Themes) 

Trial and Error 
Rational 

Small Change = 
Big Effect 

No Change 

Emphasis Upon 
Controllable Events 

Averages/Small 

Input 

Circular Causality 

Tradeoffs 

Applied Knowledge 

Balance Variables 


Note. Words identified in bold letters represent emergent-like characteristics. Regular text represents 
reductive traits. 
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Appendix B Emergent-Like and Reductive Decisions in C02FX Simulation 



0 


Trial 1 


Trial 2 Trial 3 


Figure 1. C02FX data decisions over three or more decades. No change line displays an 
emergent-like pattern with values held consistent across a minimum of three decades. 4% 
or less line displays emergent-like characteristics with minor changes of 0% to 4% 
inputted between three or more consecutive decades. 5% or more line represents 
attributes of reductive-type behavior with data input decisions varying decade-to-decade 
between 5% and 99%. 


71 





EFFECTS OF METACOGNITIVE JOURNALING ON ACADEMIC 
ACHIEVEMENT OF HIGH SCHOOL STUDENTS 


Abstract 


Katrina M. Cox 
Utah State University 


Learning Journals have been used and proven effeetive for undergraduate eollege 
students and for ninth graders in inereasing grade performanee and general 
eomprehension in a variety of fields of study. A limited number of studies have shown 
that Metaeognitive Journaling, a form of Learning Journals, is more effeetive. Three 
studies that researehed Metaeognitive Journals have weaknesses in study design. 
Additionally, no study has been found testing the effeets of journaling with students in 
grades 10 through 12. It would be benefieial to determine if Metaeognitive Journals are 
better than the other forms of Learning Journals sinee teaehers eould have a new, more 
effeetive tool for improving student grade performanee and eomprehension that is no 
more time eonsuming than other Learning Journals. Therefore, a study will be done 
among students grades 9 through 12, using three eomparative groups: (1) Control, (2) 
Metaeognitive Journal, (3) Learning Journal (Seientifie/Experimental), to test 
Metaeognitive Journals’ effeet on eourse grade performanee and final exam seores. 

Learning Journals 

A Learning Journal’s main foeus is on the learning experienee (Park, 2003). From this, a 
variety of journal forms have appeared sueh as Refieetive/Reading (Thomas & 

Barksdale -Ladd, 2000), Experimental/Seientifie, Cognitive, Affeetive, and Metaeognitive 
Journaling (MeCrindle & Christensen, 1995; Smith, Rook, & Smith, 2007) and are a 
positive influenee for undergraduate eollege students and 9**^ graders in that: 

It has been weleomed as a learning tool, and . . .offers an autobiographieal 
approaeh to learning, a way of improving knowledge and learning, a way of 
developing refleetive practiee, and a help to developing the eourse of one’s own 
learning (Park, 2003). 

They also improve student grades, understanding, and feelings of eapability in sueh fields 
as English (Mannion, 2001, O’Rourke, 1998), Algebra (Pugalee, 2004), Biology 
(MeCrindle & Christensen, 1995), Psyehology (Connor-Greene, 2000), History (Smith, 
Rook, & Smith, 2007), Geography (Park, 2003), and Reading Comprehension (Thomas 
& Barksdale-Eadd, 2000). No negative effeets beyond student eomplaints of eompletion 
time and initial understanding of the use of journals were found (Park, 2003). Metrie and 
verbal analysis showed positive effeets of their use in learning and teaehing for students’ 
pereeptions and performanee and teaehers’ methodologies. 
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Metacognitive Journals 

Metacognitive Journaling, defined by an ability to induee self metaeognitive awareness or 
. .one’s knowledge eoneerning one’s own eognitive proeess and products or anything 
related to them, e.g. the learning of relevant properties of information or data” (Flavell, 
1976) and the ability to maintain executive control over those faculties, must incorporate 
predicting, planning, revising, selecting, classifying and checking (Smith, Rook, & 

Smith, 2007; McCrindle & Christensen, 1995; Thomas & Barksdale-Ladd, 2000; 

Pugalee, 2004; and Chester, 2007) and must also address Flavell’s three domains of 
metacognitive awareness: 1) Person, 2) Task, and 3) Strategic (McCrindle & 

Christensen, 1995, pp. 169-170). All of these recommendations are best exemplified by 
McCrindle and Christensen’s questions for use in Metacognitive Journals (1995): 

1 . What did you do? {Strategic/T ask) (p. 173) 

2. How did you learn the content? (Task) (p. 173) 

3. Explain as fully as you can what strategies you used to comprehend, learn, 
and recall the information. . . [with] Reflection on the strategies used, how 
effective they were, evaluations or recommendations for their future use, and 
other reflections on their personal learning process (Strategic/Person) 

(pp. 173-174). 

Metacognitive Journaling has been found by a limited number of studies to be better than 
other Learning Journals. McCrindle and Christensen (1995) showed that students who 
used the Metacognitive Journals (M=29.4 vs. M=25.25) in the college undergraduate 
biology scored an average of 4.15 points higher on the final exam, used more 
metacognitive strategies,... [had] acquisition of greater metacognitive knowledge, ... 
gave richer, more detailed accounts of their learning process,. . .articulate[d] their 
understandings with greater clarity, [and] . . .demonstrated greater awareness of cognitive 
strategies as well as greater control in implementing strategies effectively (p.l82) 
than students using the ExperimentaFScientific Journals, “despite the fact that the [this] 
group had more opportunities to interact directly with the course content” (p.l84) and had 
similarly ranked and acknowledged the importance of various metacognitive strategies. 

Smith, Rook and Smith (2007) found that ninth grade History student course grades were 
consistently higher over a 12-week period with those using a combination metacognitive 
journal than those using cognitive journals or not journaling. This shows the 
metacognitive addition “support[s] content learning to a much greater degree than just 
cognitive or text-related” styles (p.46). 

Study Flaws 

Only three studies found tested Metacognitive Journals directly: Thomas and Barksdale- 
Ladd (2000); McCrindle and Christensen (1995); Smith, Rook and Smith, (2007). Smith, 
Rook and Smith (2007) tested cognitive (A), affective (B), and metacognitive (C) writing 
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styles in an X, A, ABC style with no eapability of eomparing the effeets of the 
metaeognitive element alone, rather, eomparisons eould only be made between the ABC 
treatment. A, or none at all (X). This study also fails to distinguish between the various 
forms of questioning/journaling they tested with no examples of what questions looked 
like, how they were used, or even formed. 

The Thomas and Barksdale-Ladd (2000) study, while providing detail in what the 
journals were eomposed of, also, used eombination treatments, but no eontrol group. The 
eombinatory effeet of all three treatments was discussed in a case study description of 
behavior, but not in a comparison of any kind via internal or external controls. They also 
fail to address major aspects of Metaeognitive Journaling by monitoring, predicting, 
hypothesizing, and judging/evaluating reading materials and text, not the manners in 
which they learned or monitored their cognitive behaviors. 

The McCrindle and Christensen study of 1995 did not use a control group. Although it 
claimed that the Experimental/ Scientific Journaling group was a control, it is still a 
Learning Journal, and, thus, affected student performance on many measures. Without 
the incorporation of a non-journaling control group to compare any result “would not be 
conclusive in itself without comparison to a control group. . .Moreover, a significant 
difference for the control group would alert you to the possibility of extraneous factors 
affecting your independent variable” (Cohen, 2001, p.305). 

Benefits of Metaeognitive Journals 

If Metaeognitive Journaling were found more effective, it would produce increased 
student performance using means that are no more time consuming than traditional 
Learning Journals (Thomas & Barsdale-Ladd, 2000; McCrindle & Christensen, 1995). 
They would theoretically “provide[s] a mechanism to increase metacognition” 

(McCrindle & Christensen, 1995) “an essential element of comprehension” (Thomas & 
Barsdale-Ladd, 2000) “central to the development of ‘expert learners’,” improve student 
grade performance and comprehension, and “potentially be seen to provide a sound 
framework for the development of ‘capable’ [students]” (Phelps, Ellis & Hase, 2001). 

These issues lead to the conclusion that a better study needs to be designed that tests and 
analyzes the singular effects of Metaeognitive Journaling, uses a control group absent any 
journaling, single treatment groups, and uses proper statistical measures, not simply a 
verbal analysis and description of effects. In total, “Lurther research is necessary to 
investigate those factors that provide for the successful implementation of a writing 
program designed to develop metaeognitive behavior” (Pugalee, 2004, p.44). 
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ASPECTS OF PROBLEM SOLVING IN CHILDREN PRE-KINDERGARTEN TO 
NINTH GRADE: FOCUS ON FUNCTIONAL FIXEDNESS. 

Michael Nehring 
University of Minnesota 


Introduction 

Technology as defined by the International Technology Education Association (2000, 
p.2) is . [a] diverse collection of processes and knowledge that people use to extend 
human abilities and to satisfy human needs and wants.” According to the American 
Heritage College Dictionary (1993), the word technology is derived from the Greek 
word, tekhnologia, meaning systematic treatment of an art or craft: “teche” meaning skill, 
and “ology” defined as science, theory, or study. 

Historical Perspective 

The formal education of technology has evolved over the centuries from apprenticeships 
to high-tech laboratories in colleges and universities. Much of the evolution can be 
attributed to the original formal education programs of the late 1800s, such as Manual 
Training, Sloyd, and Manual Arts. 

Following the European Industrial Revolution, the philosopher, Johan Heinrich 
Pestalozzi, began advocating for a reconnection of the mind and hand in education. 
Pestalozzi advocated for teaching words and meanings through the manipulation of 
objects and models (Cochran, 1970; Hostetler, 1974). Influenced by Pestalozzi, Friedrich 
Wilhelm Augustus Froebel developed the first kindergarten, where children learned 
through the manipulation of three-dimensional objects. Through his study of children, 
Froebel recognized differences in individual learning development and encouraged 
individual education (Hostetler, 1974). 

Following a similar philosophy on education, Victor Della Vos developed the Russian 
system of manual activity at the Russian Imperial Technical School of Moscow (Phillips, 
1985). Della Vos developed a new educational program for engineers based on “an 
analysis of operations, processes, and manipulative work” (Cochran, 1970, p. 2). Around 
the same time in Scandinavia, Uno Cygnaes, influenced by Pestalozzi and Froebel, 
modified the Russian system to develop the Elementary Education system in Finland. 

Otto Salomon, director of a Sloyd school in Naas, Sweden, after visiting with Cygnaes in 
1877, returned with his own modification of the Russian system. 

In celebration of the centennial of the United States, the Centennial Exposition was held 
in Philadelphia in 1876. During the exposition, educational systems from around the 
world were on display, including the Russian and Sloyd systems. Two of the most 
influential people to learn about the Russian and Sloyd systems were Calvin Woodward 
and John Runkle. Woodward developed the Manual Training School of Washington 
University in St. Fouis based on the Russian system, and Runkle developed the School of 
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Mechanic Arts at the Massachusetts Institute of Technology based on the Sloyd system 
(Cochran, 1970; Hostetler, 1974; Phillips, 1985). 

Woodward believed, when combined with literature, science, mathematics, and 
theoretical mechanics, manual training would raise the machinist to the level of a 
professional, an accomplished mechanical engineer. Woodward argued this combination 
of the traditional liberal arts and mechanical arts creates a perfect balance of knowledge 
needed for inventiveness (Woodward, 1890). Woodward and Runkle advocated for 
manual training or manual arts to be a new approach to general education, not as a new 
form of trades development. Runkle (1882) writes that, although not opposed to the idea 
of trade schools’ being included in public schools, there needs to be substantial 
justification of how it will provide special educational value. Runkle further writes the 
role of general education is to educate students generally, and the specific technique of 
their future trade should be learned in after he leaves school. 

Throughout the end of the 19**' century and into the 20**^ century, the philosophies of the 
Manual Training schools and Mechanic Arts schools continued to spread across the 
United States. Charles A. Bennett, professor and head of the manual arts department at 
Bradley Polytechnic Institute, argued with the rapid development of industry and 
industrial products the schools must assume in the role of educating the members of 
society about the materials, principles, and processes of industry. Bennett (1917) 
advocates for all schools to include the five areas of industrial arts: graphic arts, 
mechanic arts, plastic arts, textile arts, and bookmaking arts. It is also evident in 
Bennett’s writings of the discussion within U.S. society about whether manual arts is 
intended to be vocational or general and whether it is a method of teaching or a subject of 
its own. 

After the turn of the century, there was great debate over the role of vocational education 
in manual training/arts and its role in public education. In 1914, the Commission of 
National Aid to Vocational Education was established as a result of the U.S. Congress 
being split on the debate over vocational education. The commission recommended 
federal aid for vocational education, which later would lead to the passage of the Smith- 
Hughes Act in 1917 (Cochran, 1970). After the passage of the Smith-Hughes Act, 
vocational education programs were being developed throughout the country. 

While congress was debating federal funding for vocational education, the debate 
continued within academia. In response to the absence of educators on the Commission 
of National Aid to Vocational Education, Dewey (1914/1977) argued public schools 
should not be turned into pre-factories where the tax levy subsidizes the burden of skill 
development for employers. Dewey argues public schools should be developing 
industrial intelligence instead of skill development for the trades. Snedden (1915; 1977), 
in response to Dewey, argues that publicly supported vocational education is needed to 
insure that everyone can receive the training needed to be employed. Dewey responds to 
Snedden in kind, arguing that education is, by nature, vocational. However, Dewey 
objects to the idea of the limited scope of vocation used by vocational education 
advocates. Dewey argues that the type of vocational education targeted at children under 
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eighteen or twenty is a narrowly foeused view of voeation, foeused on manual labor skills 
at the expense of high order knowledge. 

With the sueeess of the Smith-Hughes Aet and the inerease in voeational edueation 
programs, the National Soeiety of Voeational Edueation merged with the Voeational 
Assoeiation of the Middle West to form the Ameriean Voeational Assoeiation in 1926 
(Coehran, 1970). Although there was a division within the Ameriean Voeation 
Assoeiation, the advoeates for general industrial edueation were unable to organize their 
own voiee until 1947, when the first annual Ameriean Industrial Arts Assoeiation 
eonferenee. 

Throughout the 50s, 60s, and 70s, a number of researehers and edueators attempted to 
develop eurrieulum guides to define what should be taught in Industrial Arts. In 1981, the 
Ameriean Industrial Arts Assoeiation published its first set of standards. Standards for 
Industrial Arts Programs. In 1983, the Ameriean Industrial Arts Assoeiation offieially 
ehanged its name to the International Teehnology Edueation Assoeiation (ITEA). In 
1985, the standards were revised and renamed: Standards for Teehnology Edueation 
Programs. In 1994, the ITEA began working on Teehnology for the All Amerieans 
Projeet, in an effort to develop standards for teehnologieal literaey. In 2000, ITEA 
published the Standards for Teehnologieal Eiteraey: Content for the Study of 
Teehnology, with a large emphasis on design and problem solving (Dugger, 2008). 

Problem Solving in Education 

Problem solving, the proeess of developing a plausible solution to an eneountered 
obstruetion, is an important eognitive ability used throughout a person’s life. The 
importanee of problem solving is reeognized by teaehers of English, Mathematies, 
Seienee, Soeial Studies, and Teehnology as evideneed by its inelusion in their 
professional assoeiation standards (International Reading Assoeiation and the National 
Couneil of Teaehers of English, 1996; International Teehnology Edueation Assoeiation, 
2000; National Council for the Social Studies, 1994; National Couneil of Teaehers of 
Mathematies, 2000; National Researeh Couneil, 1996). Despite the apparent importanee 
of the development of problem solving abilities, researeh on problem solving within the 
edueational eommunity is limited (Petrina, Peng, & Kim, 2007). 

The laek of researeh on problem solving within teehnology edueation requires the study 
of researeh in other diseiplines, partieularly eognitive development. One of the areas of 
problem solving in eognitive development that has reeeived mueh attention is the area of 
fixedness. Aeeording to Duneker (1935; 1945), funetional fixedness is a eondition where 
an individual’s problem solving ability is impaired due to a fixation on the eommon or 
intended funetion of an objeet. Brown (1989) eoneluded some forms of learning during 
analogous problems ean ereate a situation of funetional fixedness or negative transfer in 
ehildren as young as two. While other forms of learning are the same, analogous 
problems can create flexible thinking. Contrary to Brown’s (1989) findings, German and 
Defeyter (2000) concluded that children younger than six have immunity to functional 
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fixedness. Chrysikou (2006) in a study on insight problems eoneluded that the use of an 
alternative eategorization task eould reduee the effeet of funetional fixedness. 


Purpose of the Study 

The purpose of the study is to understand the differenees in ehildren’s thought proeesses 
as they develop solutions to teehnologieal problems. The speeifie aspeets of problem 
solving the study will foeus on are: the existenee of funetional fixedness in teehnologieal 
problem solving, understanding how funetional fixedness affeets ehildren’s development 
of solutions, and the differenees of funetional fixedness in ehildren pre-kindergarten to 
ninth grade. 

Statement of the Problem 

In general, little is understood eoneerning the eognitive and developmental dimensions of 
teehnologieal problem solving and, more speeifieally, there is a debate within the 
literature on the role of funetional fixedness in problem solving. 


Research Questions 

1 . Are ehildren impaeted by funetional fixedness when solving teehnologieal 
problems? 

2. How does funetional fixedness affeet ehildren’s solution development? 

3. Is there a differenee in funetional fixedness in ehildren from pre-kindergarten to 
ninth grade? 

Contributions to Technology Education 

The signifieanee of the study is to improve pedagogy and eurrieulum development within 
the field of teehnology and engineering. By having a deeper understanding of the 
eognitive development of problem solving, edueators will be able to improve 
instruetional strategies for teaehing students how to be better problem solvers. Seeondly, 
by having a deeper understanding of the eognitive development of problem solving, 
edueators will be able to improve the development of eurrieulum used to instruet students 
on problem solving. Finally, by having a deeper understanding of the eognitive 
development of problem solving, edueators will be better prepared to assist individuals in 
improving their problem solving strategies. 
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Introduction 

The current shortage of skilled workers in science and engineering makes it imperative 
that young students from all segments of our diverse society, particularly those currently 
least engaged, be attracted into these fields. Accelerating technological advancements and 
global competition create a demand for a full workforce of creative scientists and 
engineers. During this time of significant shortage, women are underrepresented in 
science and engineering. Women constitute a large untapped resource that has the 
potential to ease the urgent need for skilled workers. This study will examine whether the 
shortage of females in science and engineering is linked to possible gender-based 
differences in school-aged children’s divergent thinking, an important characteristic in 
science and engineering and a direct measure of creativity. Such an investigation has the 
potential to fill a research gap and serve as an aid in teaching and learning about gender- 
based differences in divergent thinking. 

Creativity 

Creativity is an essential skill for scientists, technologists and engineers who are at the 
cutting edge of solving problems and developing new innovations vital to industry and 
society as a whole. Creative persons and organizations are admired. Martin (2006) 
describes creativity as discovering or inventing something new, valuable, and 
purposefully made. Runco (2003) defines creativity as problem solving or thinking that 
involves the construction of new meaning. According to Guilford (1950), creativity is the 
ability to be creative and the trait most characteristic of creative people. Creative abilities 
establish whether an individual has the power to produce creative behavior to a 
mentionable degree. The study of creativity spans multiple disciplines, making its 
definition more complex. The field of psychology focuses on the individual and the 
important components within creativity such as cognitive and personality traits that are 
native to creative people. Creativity within the realm of sociology has focused on 
creativity as an environmental task (Tornkvist, 1998). Social psychology has studied the 
creativity process and its interaction within a given context. The field of psychology 
focuses on the individual and the important components within creativity such as 
cognitive and personality traits that are native to creative people. Past research on 
creativity has focused on enhancement, problem-solving, social influences, education, 
and personality. The sheer amount of research in creativity has, in turn, increased the 
rigor behind its evaluation (Runco, 2003). Years of research has brought more agreement 
and greater quality control, which helps to insure the reliability and validity behind the 
measurement of creativity leaving less room for bias and speculation. 
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Creativity is currently high in national priorities, generating summons for support from 
national science research boards (National Academy of Sciences, 2003; National Science 
Foundation, 2006). Companies are increasingly aware of the need for creative solutions 
in order to maintain their competitive edge and respond quickly to market challenges 
(Baillie, 2002). The products of creative science, engineering, scholarship, art, and design 
can bring immense benefits to society, as well as give satisfaction to their discoverer. 
Society is willing to invest in projects and programs that promise creative outcomes 
(National Academy of Sciences, 2003). History reflects a gender difference in significant 
creative accomplishments. There have been far more accomplishments, particularly at the 
highest level, by males in science, literature, arts, music, and technical development than 
women (Eysenck, 1995). Many researchers have determined factors that influence 
creativity, but the inconclusive nature of the current collection of research emphasizes the 
fact that more research is needed to understand gender differences in creativity. 

Creativity in Science and Engineering 

Creativity is associated with the highest levels of achievement in many fields, and 
certainly this is true in science and engineering. Creativity has enormous importance in 
science and engineering (Martin, 2006). Creativity is a key attribute of talented scientists 
and engineers; people are the engines of creative practice. In the fields of engineering and 
science, new systems, tools, processes, and equipment are the concrete result of creative 
acts (Tornkvist, 1998). Engineers develop numerous innovative and creative business 
solutions today (Eogal,1998). In science and engineering, creativity can result in new 
predictive theories, new materials, more efficient energy sources, and safer products. The 
list is endless. Research has shown that creative ability is held in high regard in science 
and engineering and that constraints may discourage creativity, such as the demand for 
productivity, competitiveness, and the various external pressures of resources such as 
time and money. 

Mowry (2004), in his article. The Power of Creativity , states that creativity is of vast 
importance to our economy. Creative individuals want to make breakthroughs in their 
discipline and strive to be inventive. Therefore, creativity serves to advance the 
disciplines in which a person is creative. Creativity carries the added importance of 
enhancing one’s sense of individual fulfillment. It provides engineers and scientists with 
a sense of meaning and self-fulfillment, therefore, creativity in science and engineering is 
a revolving win-win cycle that benefits industry and the individual. Mowry has praised 
our country’s development of and future plans for promoting the creative sector as an 
important step in the right direction. 

Divergent Thinking 

Divergent thinking is a direct measure of creativity and an important characteristic in 
successful advancements in science and engineering. Divergent thinking is defined as an 
idea-generating process wherein an individual is faced with problems or questions for 
which there is not just one answer (Guilford, 1950; Runco, Dow & Smith, 2006). It is the 
opposite of convergent thinking where ideas are eliminated to arrive at a single correct 
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answer, as in multiple choiee questions. Charles and Runeo (2001) stated that divergent 
thinking is indieative of one’s potential for ereative performanee. Integrating ereative 
thinking into professional knowledge to ereate new ideas is of major importanee (Hsiao 
& Liang, 2003). 

The eoncept of divergent thinking was developed in the 1950s by J. P. Guilford (Gale 
Group, 2001). Aceording to Guilford, divergent thinking is a key faetor in ereativity, and 
he associates it with four main ingredients. The first is elaboration, the ability to think 
through the details and carry them out. The second is flexibility, the capacity to think 
about a variety of approaches simultaneously. Third is fluency, the capability to produce 
a large number of ideas rapidly. The fourth is originality, the expertise to develop ideas 
different from most people’s ideas. There are many possible factors that may influence 
divergent thinking. Runeo, Dow and Smith (2006) identified memory, information, and 
experience as factors. Thomas and Berk (1981) reviewed the possibility of environment 
influencing divergent thinking. Reese, Lee and Cohen (2001) and McCrae, Arenberg, and 
Costa (1987) published research on divergent thinking and age differences in test results. 
Anxiety has also been shown to influence divergent thinking (Feldhusen, Denny, & 
Condon, 1965; Wadia & Newell, 1963). Gluskinos (1971) and Russo (2004) studied the 
influence of grade point average (GPA) and intelligence quotient (IQ) on creativity. The 
1960s and 1970s brought about an increased interest in non-cognitive (creativity) tests in 
an effort to identify gifted and talented students. With this effort came the need for a 
standardized testing method. Many researchers have created tests, the most popular of 
which is the Torrance Tests of Creative Thinking (TTCT) (Hsiao & Liang, 2003). This 
and other tests will be discussed in the literature review in the following chapter. 

Tests of divergent thinking use open-ended problems in order to allow the individual to 
come up with a variety of answers. Researchers use variations in testing instruments and 
in the scoring methods. One example of scoring measurements is the grouping of 
responses into three aspects (Runeo, Dow & Smith, 2006; Guilford, 1950). Originality is 
measured by the number of unique ideas presented. Flexibility is the number of 
categories or themes presented in the ideas. Ideational fluency is the number of ideas. 

Shortage of Scientists and Engineers 

Success in a global economy is highly dependent on the education and employment of the 
best pool of workers in the areas of science and engineering. The current shortage is well 
documented. The number of engineers produced in the United States per capita is 
proportionately low compared to developing high-tech countries, such as India and 
China. The population of the United States is about 300 million people, and it produces 
60,000 engineers each year (Wei, 2006). India has a population of 1 billion (or about 
three times that of the United States), and India produces 350,000 engineers annually, or 
six times that of the United States. China, with a population of 1 .4 billion (or about four 
times that of the United States) graduates 600,000 engineers a year. That is 10 times the 
number of graduates in the United States (Wei, 2006). Japan trains twice as many 
engineers and scientists as does the United States (Beech, 2000). Failure to produce 
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qualified workers means that the United States would be left in a position where it must 
compete abroad for qualified workers. 

Isidore (2007) reports that economists and labor market experts say that job growth and 
the economy overall would be significantly stronger if employers could find the skilled 
workers they desperately need. The deficiency of scientists, engineers, and technologists 
is likely the chief constraint on economic growth. The lack of workers skilled in these 
areas, in addition to the projected retirement of baby boomers, makes this an urgent 
problem that, without immediate attention, is certain to compound in the years to come. 
The workforce shortage in science and engineering would be problematic if needs 
remained fixed, but the huge growth in these fields compounds the difficulty. Marcus 
(2000) said that the Bureau of Labor Statistics anticipated that, during the years 2000- 
2006, the number of computer engineers needed would double. Marcus cited the National 
Science Foundation, which predicted jobs in engineering would grow at a rate triple that 
of other jobs. Numerous studies provide statistics proving that women are 
underrepresented in science and engineering. 

Women comprise approximately 50% of the population, yet, according to Science and 
Engineering Indicators (2008) women held only 26% of non-academic science and 
engineering occupations in 2005. DeBartolo and Bailey (2007) point out that women 
comprise fewer than 20% of engineering majors and stress that it is essential for our 
nation’s high-tech industries to increase the diversity of engineering graduates. As 
business leaders and policymakers seek to address talent shortages, it is becoming 
increasingly urgent to close this gap and leverage the talents of men and women. 

Reed-Jenkins (2003) states that females remain underrepresented in science, technology, 
engineering, and math careers. Female enrollment in technology-related fields is at the 
lowest level since 1985 (Treyvaud & Rounds, 2003). “Balancing the Equation” (1998), a 
press release by the National Council for Research on Women, stated that the United 
States workforce was comprised of 45% women in 1996 but only 12% of them held 
science and engineering jobs. The press release also stated that in 1999, women earned 
less than 20% of computer science degrees, and in 1996, they earned only 18% of 
engineering degrees. Today’s homogeneous male engineering teams are no longer able to 
deal with the increasingly diverse needs of the customers (Ihsen, 2005). The lack of 
diversity and the issue of women in engineering holds more and more political and 
economical relevance worldwide. To survive and thrive, science and engineering must 
draw from the broadest and most diverse pool of candidates to attract and retain the best 
skilled workers. A diverse workforce blending genders, cultures, and ages has the 
advantage of representing a wider customer base in order to translate customer 
requirements into new and useful products. Another factor in the scarcity of female 
scientists, engineers, and technologists is the dropout rate of women already employed in 
the field. Women tend to abandon full-time work at a higher rate than men, but this 
phenomenon is far greater in these fields (Hewlett, Euce & Servon, 2008). Many factors 
such as confidence, interests, social influences, perceptions, efficacy, desire to help 
others, physical abilities and characteristics, have been identified as potential negative 
influences on women and their lack of participation in these fields and the reasons they 
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are leaving these fields after entering them (Jaeklin 1989; Linn & Hyde, 1989). 
Identifying these factors has proved beneficial, but we are still struggling toward 
increased diversity in many fields, particularly science and engineering. 

Current research points to perceptions and stereotypes as the greatest obstructions to 
young females becoming interested and entering the fields of science and engineering. 
Perceptions have obvious implications that have contributed to low numbers of female 
participation in technology and engineering. Research shows that females view 
engineering and scientific fields as “geeky” (Muller, 2002). 

Many girls are turned off by the thought of a career in technology. They are 
haunted by the image of nerdy male co-workers drinking Red Bull, eating 
Twinkles and having meaningful relationships with their computers. Sure, we 
know it’s a cliche, but to kids— and especially young girls— image is everything. 
(Woodka, 2001, Introduction section, I) 

Based on the current research helping to identify the shortages in science and technology, 
many programs have been developed to reduce this insufficiency. 

President Bush’s American Competitiveness Initiative and the Democratic Innovation 
Agenda are two programs assembled to increase female participation through school 
funding, scholarships, and grants in science, technology, engineering, and math (STEM). 
Hundreds of programs, publicly and privately funded, have been implemented in 
response to this national deficit. Despite the wide recognition of the problem and the 
programs to intervene, participation of women in STEM is still an issue. Further 
investigation is necessary to understand the fundamental reasons. Are there other factors 
beyond stereotypes and societal norms that restrain women’s involvement? Do innate 
differences between males and females play a larger role than is currently understood? 

Statement of the Problem 

There is a shortage of scientists and engineers at a crucial time when technological 
innovation depends on the involvement of our nation’s best and brightest, representing all 
segments of our diverse society. Women comprise approximately 26% of the college 
educated workforce in science and engineering occupations (Science and Engineering 
Indicators, 2008). Sanders (2005) stated that women’s lack of participation can only be 
measured in jobs not fdled, problems not solved, and technology not created. Engineering 
must attract young people who are seeking stimulating and creative work (Wulf, 1998). 
“Diversity is the gene pool of creativity” (Wulf, 1998, p. 23). Creativity is at the heart of 
engineering and science and is essential to scientists and engineers who are responsible 
for developing many of our most innovative and creative business solutions today (Fogal, 
1998). Divergent thinking is a well accepted component of creativity (Charles and Runco, 
2001) and is central to its measurement. Understanding creativity and divergent thinking 
has the potential to shed light on the underrepresentation of women in science and 
engineering. Few studies have been conducted which analyze creativity in 
underrepresented groups and most have revealed contradictory findings (Matud, 


86 



Rodriguez, & Grande, 2007). Limited researeh has been eondueted to determine whether 
there are fundamental differenees between boys and girls in the area of ereativity and its 
key eomponent, divergent thinking. Divergent thinking is a eritieal dimension of 
inventiveness in seienee and engineering related ereativity. 

The Study 

This study will eompare gender and age differenees in divergent thinking among middle 
sehool and high sehool students in the Midwest using the figural Torranee Test of 
Creative Thinking (TTCT). Approximately 100 students from middle and high sehool, a 
total of 200 partieipants, will partieipate. Results will be seored and analyzed by the 
researeher. The middle sehool students and the high sehool students attend publie sehools 
where they are engaged in the engineering eurrieulum option offered at the sehools. The 
middle sehool aets as a feeder sehool for the high sehool pre-engineering program. 
Partieipants will be given the Figural Torranee Test of Creative Thinking, a measurement 
of divergent thinking. Results will be analyzed in an attempt to determine whether there 
exist gender-based or age-based differenees in divergent thinking: 

Research Questions 

1 . Are there gender differenees in: 

a. Originality of responses? 

b. Flueney of responses? 

e. Abstraetness of titles in the responses? 

d. Elaboration of the responses? 

e. Resistanee to premature elosure? 

2. Do age differenees exist in: 

a. Originality of responses? 

b. Flueney of responses? 

e. Abstraetness of titles in the responses? 

d. Elaboration of the responses? 

e. Resistanee to premature elosure? 

Variables and Their Measurements 

Eive main independent variables are present in this study of divergent thinking. 
Originality'. Responses are measured for originality using norm refereneing. The tested 
individuals’ responses are eompared to their peers. Eor example, if a response has been 
given by only 5% of the partieipants, it reeeives one point. If the response has been given 
by less than 1% of all respondents, the answer gets 2 points. The points are then totaled, 
and the higher the seore, the more ereative the individual. Fluency. Elueney is rated as 
the overall number of responses given to a question. Previous researeh has pointed out 
that there have been eases in whieh a partieipant’s responses eould be eonsidered repeats. 
Some answers that are expressed differently ean have the same translation. In seoring the 
responses, the rater must pay elose attention to avoid eounting duplieate answers more 
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than once in order to eomply with the TTCT streamlined scoring guide. Abstractness of 
titles: In order to measure the abstraetness of titles, also referred to as flexibility, all 
responses are eategorized. The number of eategories measure one’s flexibility. To elarify, 
if a participant is asked to name things with wheels, and their responses were a ear, a 
truek, a bike, and your mind, they would get a flexibility seore of two. One point is for 
responses that fit into the eategory of transportation, and the seeond point is for the non- 
transportation eategory, “your mind”. Elaboration: This is the level of detail presented. 
The responses will be given no points for no detail and one or more points for details 
within the response. Resistance to premature closure: This variable measures whether 
the respondent kept an open mind while proeessing information, a requirement of 
divergent thinking. It should be pointed out that the nature of ereativity and the reliability 
of eurrent measurements are still under debate by many, even after 50 years of work in 
the area (Russo, 2004). The lack of agreement in this area is often attributed to the 
multidimensional nature of ereativity. It is thought, however, that divergent thinking and 
its measures, originality, fiueney, abstraetness of titles or flexibility, elaboration, and 
resistanee to premature elosure, are vital to the study of ereativity (Torranee, 1981). 
Chapter 3 will provide more information on eaeh of the faetors and the measures used in 
seoring. 
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DESIGN TWICE, BUILD ONCE: TEACHING ENGINEERING DESIGN 
IN THE MIDDLE AND HIGH SCHOOL CLASSROOM 

Shawn Jordan and Nielsen Pereira 
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Abstract 

Hands-on learning experiences and interactive learning environments can be effective 
tools for teaching K-12 students. Design, in essence, is an interactive, hand-on 
experience. Engineering design can be taught in the classroom using hands-on projects, 
such as designing and building Rube Goldberg-style machines that complete a task in 
complex ways. Rube Goldberg activities serve to teach design, promote creativity, and 
prove opportunities for hands-on problem solving, in addition to giving students 
experience working in cooperative teams. In turn, these experiences could encourage 
students to consider engineering as a possible future career. 

This paper explores preliminary findings from data collected during the authors’ recent 
experience teaching a group of fifteen gifted and talented students in grades 4-6 enrolled 
in a 6-week Saturday enrichment program to design and build Rube Goldberg machines. 
A scaffolded engineering design process was used to guide groups of 3-4 students 
through the project. Students took on defined roles in order to promote teamwork. This 
paper will present both aspects of the course that worked well and areas for improvement, 
in addition to surprises encountered along the way. Finally, thoughts on migrating the 
class to high school students will be discussed. 

Introduction 

Rube Goldberg activities have been used in classrooms throughout the country for many 
years. Despite their use, little research has been done to understand how they should be 
used for maximal benefit. The class described in this paper as a “report from the field” 
and the associated research in progress are unique because they apply an engineering 
design process to the teaching of Rube Goldberg machines. In the education space, this 
study incorporates participants who have ADHD, anxiety disorders, and mild autism. 

Class Background 

Saturday enrichment programs can be an alternative strategy for gifted children who need 
more advanced content in a specific field. Children are usually taken to programs outside 
of regular schools, such as university camps. This type of programming offers several 
benefits to gifted students, such as exposure to advanced content in diverse subject areas, 
highly qualified instructors, and interaction with like-ability peers in a learning 
environment where the students feel safe to be themselves (Davis & Rimm, 2004). 

Special gifted programs can also provide affective gains in participants’ self-esteem, self- 
efficacy, and academic motivation and these gains may lead to success in school after 
attending such programs (Olszewski-Kubilius & Grant, 1996). 
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Super Saturday is an enrichment program for children in grades P-8 that takes place at 
Purdue University’s main campus, in West Lafayette and is sponsored by the Gifted 
Education Resource Institute (GERI). Super Saturday and similar programs meet some 
needs of gifted children that schools do not usually deal with and those include 
achievement of basic concepts and skills to their maximum; adequate pace and level of 
activities; improvement of self-awareness and of the understanding of one’s own ability 
level, interests, and needs; increase in independence; self-direction and discipline in 
learning (Eeldhusen & Wyman, 1980). Parents of students in the class are informed that 
the classes are challenging and that students who do not show high-ability or interest in 
the topic of the class may not be able to benefit as much from the program. 

Class Goals 

This class sought to teach engineering concepts and meet the needs of high- ability 
students. Our first specific goal was to have students work together in small groups in an 
effective way. The students’ second goal was to understand and be able to apply an 
engineering design process to a problem. Their third goal was to understand the science 
and engineering concepts used in Rube Goldberg machines. The last goal was to build the 
machine they designed. 

Class Outline 

The course had a total of eighteen hours divided in six different Saturdays. During the 
first class, we had a brief introduction of the class, and students filled out a brief pre- 
assessment and an interest inventory. We had an introduction to mechanics concepts and 
some brainstorming aimed to encourage students to exchange ideas freely and to learn to 
work in groups. Students, also, worked on the design of the first module. We initially had 
the goal of having a structure for the class so that groups of students would work on a 
module at a time and would integrate their modules and test them before final 
presentations. Our initial plan was to have them create one module in about 3-4 hours and 
always to follow an engineering design process, that is, students would have to first, 
design, then build, and, finally, test their modules. During classes 2 and 3, students 
worked on the design of the first module, built, and tested this module. We had a brief 
introduction to electricity, magnetism, fluid mechanics, some brainstorming and design 
for the second module. 

The fourth class was used to finish the design, build, and test the second module. Most of 
the groups could only finish their second module by the end of class 4, so they had to use 
class 5 for building and working on the integration of modules and testing of their 
machines. During class 6, groups had to reassemble their machines and test them again 
before final presentations to family members and classmates. 
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Participants 

The participants in this study were fifteen students in a Saturday enrichment class in 
grades 5-6. Students in the program, typically, attend local schools in the greater 
Lafayette area, but some come from other cities in Indiana. Four of these students were 
part of a GERI initiative to bring low-SES children to gifted programs. Project HOPE, 
and that made our sample even more diverse and representative of the different types of 
communities in Indiana, such as rural, suburban, and urban locations. There were twelve 
boys and three girls. 

Research Questions 

This study seeks to explore the following research questions: 

1 . How effective is teaching an engineering design process to 5th and 6th grade 
gifted students? 

2. How do group interactions influence design process outcomes? 

3. How can an engineering design process be taught to 5th and 6th graders? 

4. How do design ideas generated by 5th and 6th graders evolve across different 
stages of the design process? 

5. What effect do design artifacts have on final products? 

Methods of Data Collection 

A variety of data was collected during the class. A student interest inventory, conducted 
on the first day of class, provides a glimpse into what the students like to do. A pre- 
assessment, also conducted on the first day of class, was used to determine the 
background each student had with Rube Goldberg and related activities. The 
brainstorming processes had artifacts in the form of written lists and sketches that were 
collected. Design sketches were also collected, as well as the final posters that teams 
made including sketches of the final machine design and a written sequence of steps. 
Pictures were taken throughout the class, and videos of the final demonstrations were 
made. Einally, the two instructors kept reflective teaching journals. These data were 
collected and electronically stored (e.g., the sketches were scanned) in a master file 
during the class. 

Preliminary Results 

Eormal analyses of the data are in progress. An early look at the reflective teaching 
journals reveals some interesting trends. Eirst, teamwork was a challenge for the students. 
Gifted children often prefer working alone than with others, and this class was no 
exception. The notion of individual roles in teams was introduced in order to facilitate 
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better teamwork, but the students did not adopt the roles well and usually fell baek into 
eaeh person working independently. 


Second, the design process was difficult for the students to use. The students wanted to 
come in and build the entire time without doing any design work. They, also, did not 
want to document anything they did before, during, or after they built it. 

Finally, maturity of the children was an issue throughout the class. Many of the students 
had trouble focusing on the activities for long periods of time without getting distracted. 
This did not appear to be due to a lack of interest, but rather a lack of attention span. 
Additionally, structure was a problem. The environment was reasonably structured, but 
the students had trouble remaining within the structural bounds of the class. 

Recommendations and Remarks 

For those considering implementation of a hands-on project like this in a high school 
environment, we definitely recommend a strong connection between research and 
practice, with an emphasis on informing each from the other. Long term goals do not 
seem to hold the attention span of kids as much as shorter, moderated goals. Checkpoints 
can help give students something to work toward. An engineering design process may be 
somewhat tedious for students to use but can be motivated by using problems that are too 
difficult to solve without design. 

The Next Iteration 

The class will be offered in a summer residential program for high-ability learners in the 
summer of 2008 with some improvements based on the first experience. The participants 
will be 10-14 students who were in 7th or 8th grade in the 2007/2008 academic year. 
There will be a total of 30 contact hours, and some of the changes we plan include 
providing more scaffolding to the groups. The first project will involve building a pre- 
designed machine. The engineering design process will be introduced only after students 
have finished the first project. For the second project, groups will be given input and 
output specifications for their modules and will have to design and build these and later 
integrate them in a ‘whole-class machine.’ The third project will challenge students to 
complete the whole engineering design process and brainstorm, design, and build each 
module, and then integrate these. 

Another change will be for each group to work on a different module at a time. These 
modules will, then, be integrated in ‘whole-class projects’, thus, providing students with 
short-term goals and also allowing them to work as a class toward a common goal. 

Conclusion 

This work in progress seeks to further illuminate the question of how to teach an 
engineering design process to kids through the design of Rube Goldberg machines. 
Preliminary results indicate that kids have trouble working in teams, applying a design 
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process, and demonstrating suffieient maturity to foeus and manage their own sehedule 

toward an abstraet goal. 
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ACADEMIC PERFORMANCE AS A PREDICTOR OF STUDENT GROWTH 
IN ACHIEVEMENT AND MENTAL MOTIVATION DURING AN 
ENGINEERING DESIGN CHALLENGE IN ENGINEERING 
AND TECHNOLOGY EDUCATION 

Nathan Mentzer 
Utah State University 


The purpose of this correlational research study was to determine if a student’s academic 
success is correlated with: 1) the student’s change in achievement during an engineering 
design challenge; 2) the student’s change in mental motivation toward solving problems 
and critical thinking during an engineering design challenge. Multiple experimental 
studies have shown engineering design challenges increase student achievement and 
attitude toward learning, but conflicting evidence surrounds the impact on higher and 
lower academically achieving students. 

A classroom was chosen in which elements of engineering design were purposefully 
taught. Student participants represented a diverse set of academic backgrounds (measured 
by GPA). Participants were measured in terms of achievement and mental motivation at 
three time points. 

Multilevel modeling techniques were employed to identify significant predictors in 
achievement growth and mental motivation growth during the school year. While initial 
analysis is in progress at this publication date, interesting factors have evolved. 
Achievement change is not significantly related to GPA. Mental motivation change is 
generally not related to GPA. 

The Standards for Technological Literacy (STL), include engineering in general, and 
engineering design, specifically. This paradigm shift to include engineering content in 
technology education curricula demands the field identify successful approaches to 
teaching engineering at the high school level. “Design appropriate for technology 
education is characterized by open-ended problems where the designer bridges the gap 
between past experiences and the current problem to be solved; one method of achieving 
this transition is through engineering design challenges” (Lewis, 2005, p. 49). 

In the past 15 years, 13 studies have been published which focus on the efficacy of 
engineering design challenges. Student achievement was unanimously improved at the 
conclusion of each engineering design challenge. However, upon closer inspection of the 
disaggregated results, a troubling conflict surfaces. Cantrell, Pekca and Ahmad (2006) 
concluded that engineering activities reduced achievement gaps of most ethnic minority 
groups. In contrast, Weir (2004) considered an academic upper and lower half in a 
university engineering course. Weir’s conclusion that engineering challenges actually 
extend the achievement gap by improving the academically successful students 
disproportionately to lower achieving students must be further investigated due to its 
potentially damaging educational impact. 
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Nine of the 13 studies considered attitudinal measures, and each showed improvement. 
However, validity and reliability of these instruments was questionable. For this study, 
the validated California Measure of Mental Motivation (CMS) was used. 

Technology Education has typically been a curricular area where a broad range of 
students can be successful, from the academically gifted students to the academically 
challenged students. Engineering, traditionally reserved for the academically elite 
students, will be intersecting a broad cross section of the general education populace. 

This interface necessarily includes a subset of students who are challenged by 
traditionally “academic” material. Therefore, the purpose of this study was to investigate 
a correlation of a student’s general academic history with achievement and mental 
motivation during an engineering design challenge. 

A classroom was identified in which a physics teacher partnered with a technology 
education teacher to infuse and apply engineering concepts. This junior level, high school 
course includes an academically diverse array of students and provides a semester long 
engineering design challenge. 

In order to address this research question, a repeated measures correlation study was 
conducted in which data were gathered on student achievement and mental motivation. 
Trends and changes during the year were compared to a general indicator of each 
student’s academic history. 

Data analysis was conducted using multilevel modeling techniques. Achievement scores 
did not significantly change over time. Statistically significant predictors in this model 
are special education status, GPA in previous science courses, and the CMS subscale of 
creative problem solving. Special education students tended to perform under their peers. 
Students who maintained a higher science GPA and students scoring higher on creative 
problem solving tended to demonstrate an increase in achievement scores. 

As published with the CMS, a student scoring high in mental focus is diligent, focused, 
systematic, task-oriented, organized and clear-headed. Mental focus scores increased 
significantly over time. Statistically significant and positively correlated main effects are 
GPA in math and science and time. A significant negative interaction was discovered 
between time and science GPA. 

A student scoring high in creative problem solving has a tendency to approach problem 
solving with innovative or original ideas and solutions according to the CMS. Creative 
problem solving scores increased significantly over time. Statistically significant and 
positively correlated main effects are GPA in science and time. 

As published with the CMS, a student scoring high in learning orientation is motivated by 
the desire to increase knowledge and skill base. A high score in cognitive integrity 
represents motivation to use thinking skills in a fair-minded fashion, seek the truth and be 
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open-minded. Neither learning orientation nor cognitive integrity scores changed 
significantly over time or had any statistically significant main effects. 


A student scoring high in scholarly rigor was inclined to work hard to interpret and 
achieve a deeper understanding of complex or abstract material according to the CMS. 
Scholarly rigor scores did not change significantly over time. The statistically significant 
and positively correlated main effect was GPA in science. 

Student achievement is significantly correlated with science GPA, but not math or 
communication GPA. Changes in achievement score over time are not significantly 
correlated with science, math or communication. Mental Motivation was measured by 
five subscales. Mental focus was correlated with both math and science GPA. Mental 
focus increases over time were negatively correlated with science GPA indicating that the 
initial score differential (between higher and lower science GPA students) was decreased 
over time. Learning orientation and cognitive integrity were not correlated with GPA. 
Creative problem solving was correlated with science GPA, but gains over time were not 
correlated with GPA. Scholarly rigor was correlated with science GPA, but change over 
time was not correlated with GPA. 

Knowledge of a student’s status as an underrepresented population in engineering and 
technology education improved model fit statistically for each outcome considered. 

While this predictor significantly improved the model, it was not a statistically significant 
predictor. Chance alone may be responsible for the necessity of this predictor in the 
model, or a large variance may be masking discovery of an important correlation. 

The field of technology education embraces the importance of technological literacy and 
caters to a diverse audience of student learners. Engineering design is an important aspect 
of technological literacy. An understanding of science (measured by GPA) is positively 
correlated with student achievement in this course. Most importantly, a change across 
repeated measures shows no significant correlation to math, science or communication 
GPA. This indicates that students will not be proportionately advantaged or 
disadvantaged over time by their previous academic successes. 

Mental motivation subscales demonstrate that GPA in science correlates with the scores 
in mental focus, creative problem solving and scholarly rigor, while math correlates only 
with mental focus. Each of the five subscales demonstrated a mean increase from pre- to 
post-testing with mental focus and creative problem solving being statistically significant. 
This increase over time was generally not correlated to GPA in math, science or 
communications. This indicates that students who are struggling academically and their 
higher achieving peers have an equal tendency to increase their mental motivation scores. 
In the case of mental focus, the impact of GPA in science was negatively correlated with 
time which means that the gap between low and high achievers is reduced over time. 

Secondary technology education teachers should infuse engineering into their curriculum 
understanding the concepts surrounding the application of engineering design. This study 
indicates students of varying academic backgrounds will each experience a level of 
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success which may vary, but change over time will not serve to inerease the achievement 
or motivation gap. 

Further researeh should be eondueted to better assess student aehievement ehange over 
time. This study showed no signifieant gains in achievement and, therefore, eonclusions 
and implieations on achievement change should be eonservatively eonsidered. 

Additional researeh should investigate potential eorrelations of general aeademic sueeess 
with aehievement and mental motivation for underrepresented populations. In each 
outcome, it was important to eontrol for this status, but differenees were not signifieant. 
This reeurrent theme neeessitates further investigation. 
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A STUDY OF FACTORS AFFECTING CAREER DECISION-MAKING SELF- 
EFFICACY AND ENGINEERING RELATED GOAL INTENTIONS AMONG 
AFRICAN AMERICAN HIGH SCHOOL STUDENTS 

Chandra Austin 
University of Minnesota 


Introduction 

The idea of disproportionate education is not recent. It has been around for decades, 
stemming from what some call the involuntary citizenship as a result of slavery. “In the 
past, black Americans, for example were given inferior education by formal statues in the 
South and by informal practices in the North” (Ogbu, 1987). Parents were taught by 
oppressors to say to their children that there were certain spheres which they should not 
enter because they would have no chance for development (Woodson, 1990). However, 
during an era of segregation came a well-known case entitled Brown vs. Board of 
Education, which gave hope that racial integration would correct the inequalities and 
underachievement of minority students (Freeman, Brookhart, & Loadman, 1999). But in 
this new century, while schools have become integrated as bodies, the content students 
are offered is still in a segregated state, as are expectations about student destinations. In 
this research study this will be done by looking at factors currently influencing 
educational aspects, but I must first start with the historical roots. 

History of African American Education 

The idea of institutional racism that stems from slavery is believed currently to affect the 
educational system. Institutional racism can be easily defined as the hierarchical 
conception of intellectual ability (Denbo & Beaulieu, 2002). Institutional racism has the 
inherent ability to promote the notion of inferior mental ability and often results in 
practices such as academic tracking. When tracking is done, it is often as a result of the 
focus on individual and cultural characteristics of students rather than the ways that the 
social system structures academic success and failures for varying groups. 

Shaffer, Ortman, and Denbo (2002) stated that to understand fully African American 
student achievement, it is essential to take into consideration the historical context of 
racial oppression and current conditions in schools. Since there has been, in reality, a 
divide in ethnic and racial groups, the longing affect that students have suffered must be 
interpreted. Longstreet (1978) notes that ethnic groups are unique in several different 
ways: verbal and nonverbal communication, social value patterns, and intellectual modes. 
With this knowledge, one can conclude that it is virtually impossible to educate different 
ethnic groups in the same manner (Davenport, 1981). However, even within ethnic 
groups, there appears to be some disparity with education. Urban students have less 
access to a variety of educationally important resources, such as small class sizes, highly 
qualified teachers, computers, advanced level courses, and other curriculum supports 
(Darling-Hammond, 1997). Middle-class African American students are an example of 
students who, although they are equipped with more resources, may feel that excelling in 
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school, while their lower ineome eounterparts are not, is betraying their true identity 
(Shaffer, Ortman, & Denbo, 2002). Regardless of the disparity, Apple (2004) agrees that 
sehools eontribute to the imbalanee of power in soeiety by eommunieating soeiety’s 
eeonomie, politieal and eultural knowledge to students. 

In this study, I hope to explore why more Afriean Amerieans are not partieipating in 
engineering and offer solutions that may attend to the speeifie barriers these students 
pereeive. This will be done by looking at a broad prospeet of edueational influenees, then 
narrowing the speeifie field with the speeifie population. 

Engineering 

In the eontext of engineering edueation, it is worthwhile to look at the views of Booker T. 
Washington and William E. B. DuBois. Washington and Du Bois reeognized the value of 
edueation and reeognition of the neeessity of blaek partieipation in skilled trades 
(Wharton, 1992). The eontroversy eame in the differenees of their approaeh to leadership 
and dominanee, and philosophies regarding blaek higher edueation. Booker T. 
Washington was born a slave and did not begin his formal edueation until after the Civil 
War when he was freed. However, in relation to voeational edueation, in many instanees, 
Washington foreshadowed the well-known John Dewey by two deeades (Gordon, 2008). 
Washington thought in order for people to gain satisfaetion in edueation, they must give 
serviee to other. One way he implemented this belief as president of Tuskegee Institute 
was to require students to do some form of manual labor as a part of the eurrieula. 
Unfortunately, many believed that mueh of Washington’s views forestalled the 
involvement of Afriean Amerieans in engineering by almost three deeades. This belief 
was held due to the inferior nature Washington promoted to Afriean Ameriean youth. He 
eondemned bright, young minds to voeations beneath their ability, thereby, reinforeing 
inferiority (Wharton, 1992). Washington’s edueational philosophy was not designed to 
produee individuals who would be able to eompete with whites for jobs, but to instill in 
Afriean Amerieans the glory of being manual laborers (Gordon, 2008; Wharton, 1992). 

William Edward Burghardt DuBois was born in Massaehusetts to a Ereneh mother and 
Haitian father. Unlike Washington, he was born a free man. Their different baekgrounds, 
their lives and edueations, ean be said to have been the souree of their differenees on 
edueation for Afriean Amerieans. Du Bois denouneed the work of Washington. He did 
this by promoting blaek worth, giving hope and inspiration to those who wanted to 
pursue engineering as well as other subjeets. Du Bois ereated a notion referred to as the 
“talented tenth”. This was the small pereentage of blaeks who were endowed with talents 
and brains to lead the raee to self-suffioieney. He insisted that the eollege trained elite 
eould lift the lower elass. He felt sueeess would eome from the development of mental 
faeulties. 

Contrary to the faet that Booker T. Washington and W. E. B DuBois held different beliefs 
about voeations and engineering, both ean be viewed to have enhaneed the Afriean 
Ameriean population in their own way. Engineering is believed to be a voeation whieh 
eombines the eharaeteristies of seienee, art, and business. It involves knowledge of the 
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forces and materials of nature, an understanding of men, and an understanding of 
eeonomie and soeial relations (Dowing, 1935). But the history of engineering is slightly 
different from what is eonsidered the early voeations. The early eurrieulum in Ameriean 
eolleges of engineering was still eonsidered an alternative to what was viewed as the 
traditional elassieal diseipline. In early Ameriea, unlike the fields of medieine and law, 
engineering edueation had never been under the exelusive domain of a professional 
group, the eurrieulum ereated strietly by edueators (Grayson, 1980). Therefore, those 
who were not engineers and knew little of the eontent in whieh it entailed had enormous 
input to shaping the field of engineering. As a result, the early engineering eurrieula 
eontained little teehnique of engineering praetiee. Prior to WWI, few opportunities 
existed for blaeks to work in engineering fields. But there were in existing Afriean 
Ameriean sehools that edueated some. Speeifieally, the first blaek engineers to graduate 
from blaek schools came from Howard University, North Carolina Agrieultural & 
Teehnieal State University, and Hampton University. More importantly, these sehools are 
still revered for eontinuing to edueate Afriean Ameriean students in field sueh as 
engineering. 

The importanee of edueating Afriean Amerieans was stated in the 1930’s and is still 
being restated well into the 2U* eentury. The U.S. has lost some teehnologieal ground and 
will eontinue to do so if everyone is not provided an equal opportunity to advanee in all 
areas. Some have gone so far as to say that Afriean Amerieans are not well edueated in 
the field of engineering beeause the information they are obtaining is not relevant. 
Woodson (1933) eoneurs by stating that, “Afriean Amerieans do not need someone to 
guide them to what persons of another raee have developed, they must be taught to think 
and develop something for themselves”. Or, some may go further baek to say that, during 
slavery, Afriean Amerieans developed a souree of dependeney, and now it is their job to 
grow out of it. Regardless of what the souree of this disparity in edueation, all would 
agree that it is something that needs to be reeoneiled. As stated in the previous seetions, 
the idea of eurrieulum, teaeher influenee, ete. are not new eoneepts. These are just 
eoneepts that have not been readily adapted as they relate to Afriean Ameriean students. 

Rationale 

The National Seienee Foundation (NSF) in 2006 reported that 5.2% of engineering 
degrees were awarded to Afriean Amerieans. Although sehools today are not deliberately 
designed to aehieve elassist or raeist ends, through researeh, most sehools and edueators 
are often found to be eontradieting (Apple, 2004). Researeh shows that the eurrent 
edueation of Afriean Amerieans is unequal to that of white students (Norman et ah, 

2001). However, there is an absenee of literature foeusing on the field of engineering, 
speeifieally. By studying the eareer deeision-making, self-effieaey and engineering goal 
related intentions these students have as a result of a variety of faetors, I hope to gain 
insight as to why Afriean Ameriean students are absent from the field. From this insight, 
the edueational field will be able to gain a better understanding as to how to enhanee the 
edueational efforts in assisting with the preparation of Afriean Amerieans in the field of 
engineering. In addition, some eneouragement faetors may be obtained based on what the 
students view as their pereeived needs in order to prepare for sueh a field. Lastly, if the 
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factors that influence engineering eareer deeisions are identified, edueators will better 
understand how to foster and develop a eulturally responsive environment for Afriean 
Ameriean students. With the birth of a different environment, students may be more 
inelined to engage in the study of engineering. Knowing how to educate and mediate 
eertain variables that ean eneourage entranee into engineering will eontribute to an 
inerease of minorities, partieularly Afriean Amerieans. 

Statement of the Problem 

This study intends to fill a gap in knowledge as to why Afriean Ameriean students are not 
entering engineering. Little is known about the engineering eareer deeision-making and 
related goal intentions among Afriean Ameriean students. Speeifieally, predictors of 
career decision making self-efficacy and engineering goal intentions are not well 
researehed. Currently, there is a laek of studies of students’ perspeetives of pereeived 
barriers impaeting their ehoiee of a eareer in engineering. Underutilization of minorities 
in scienee and engineering is a problem of national priority (Leslie et al, 1998), and if 
America is striving to maintain its global competiveness in the world, we must try to 
edueate as many people in different areas as possible. 

Theory 

There are a number of theories suggesting possible predieting faetors of adoleseent eareer 
deeision making. The first theory is the self effieaey theory developed by Albert Bandura. 
Bandura’s (1977) self-effieaey theory makes the assumption that personal self-effieaey is 
based on four major sourees of information; performanee aeeomplishments, viearious 
experiments, verbal persuasions, physiologieal states. Bandura’s theory is valuable 
beeause a variety of studies have found a link between self-effieaey and the ability of 
adolescents to make deeisions about careers (Lent et ah, 1986; Lent & Haekett, 1987; 
Taylor & Betz, 1983). A seeond and important theory that will be used and is a by- 
produet of the Soeial Cognitive Theory (Bandura, 1986) is the Soeial Cognitive Career 
Theory (SCCT) developed by Lent, Brown, and Haekett (1987). This theory is essential 
beeause its main purpose is to eonstruct conneetions from other theories whieh will then 
identify a better explanation linking those variables that may infiuenee eareer 
development (Brown et al, 1996). A third theory that will be used in this study is Super’s 
developmental self-eoneept theory of voeational behavior. In this theory. Super (1954) 
asserts that people attempt to apply their self-eoneept by ehoosing a eareer that permits 
self-expression. He goes on to make the elaim that a person’s behavior refleets that 
individual’s partieular life stage whieh is not the same in adoleseenee and adulthood 
(Osipow, 1983). Super’s theory is vital because self-eoneept and voeational development 
have proven to be important faetors and could influence the ereation of more eompatible 
currieulum for Afriean Ameriean students. A fourth and last theory that will be 
refereneed is Holland’s (1959) Career Typology Theory of Voeational Behavior. In this 
theory, Holland holds that eareer choiees represent an extension of an individual’s 
personality (Osipow, 1983; Sauermann, 2005). He states that people identify their views 
of themselves by an oecupational title. Holland’s theory is influential to this study 
partieularly beeause of the suggestion that he makes in how people ehoose eareers. If, in 
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fact, people ehoose eareers where they believe they will be surrounded by people like 
themselves, the edueation field needs to develop a strategy to attraet Afriean Ameriean 
students. The eonelusion eould be easily drawn that Afriean Ameriean students do not 
enter engineering beeause they do not feel that sense of likeness. Although these theories 
are regarded as important, few of them have been applied to minority populations, 
speoifieally, those foeused on Afriean Ameriean youth. Sinee Afriean Ameriean students 
are the population whieh seem invisible in the field, the relationship between theory and 
reality needs to be established in the field of engineering. 

Important Studies 

Some of the important literature that will be used to develop variables are laid out in this 
seetion. However, the variable development is not strietly limited to these existing 
studies. Navarro (2007) used a modified version of the Soeial Cognitive Career Theory to 
examine whether soeioeognitive variables explained math/seienee goal in Mexiean 
Ameriean middle sehool students. Although this study was done at the middle sehool 
level, it is still benefieial to the eurrent study in that it observes an underrepresented 
minority group. Also, Navarro et al. more speeifieally found that within this population, 
math/seienee interest and goals eould be predieted by math/seienee self-effieaey and 
outeome expeetations. Fouad and Smith (1996) also eondueted a study using middle 
sehool students and found interest had a relationship with self-effieaey, outeome 
expeetations and intentions. At the end of their study, they noted that more researeh is 
needed to test the infiuenee of raee and ethnieity as a variable affeeting these same 
variables . Gushue (2006) evaluates the relationship of ethnie identity to eareer interest 
and outeome expeetations among Latino students. This was identified as a key study 
again beeause it involved minority students but also beeause it studies eareer deeision- 
making self-effieaey as a key variable. He found that ethnie identity had a direet and 
positive relationship with eareer deeision-making self-effieaey. A eonneetion between 
raee/ethnieity and eareer aspirations and deeision making was also found by Flores et. al, 
(2006) and Kenny et al.( 2007). 

Hargrove et al. (2002) explored the relationship between family interaetion patterns and 
voeational identity and eareer deeision-making self-effieaey. The researehers found that 
family interaetion patterns play a signifieant role in the promotion of self-eonfidenee as it 
relates to eareer planning. The study also found that family interaetion patterns play a 
role in the formulation of eareer goals. Lent (1991) and Betz and Haekett (1983) are long 
time researehers of soeioeognitive variables. But, they speeifieally did a study that 
examined the relationship between math self-effieaey and seienee-based eollege majors. 
From the results, they were able to eonelude that math self-effieaey was significantly 
related to ehoosing a seienee based major. They went on to postulate that this seleetion of 
major direetly resulted in the eareer ehoiee within the same field. 
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Research Questions 

There are three dominating researeh questions that guide this study. The questions will be 
evaluated using quantitative methods. 

1. To what extent are exogenous (family relations, demographie faetors, sehool 
faetors) influential on the dependent variables? 

2. To what extent are endogenous (interest, ability, ethnie identity, and math/seienee 
self-effieaey) variables influential on the dependent variables? 

3. To what extent are exogenous and endogenous variables influential on eaeh 
other? 

Acknowledgement 

This material is based upon work supported by the National Scienee Foundation under 
Grant No. 0426421. Any opinions, findings, and eonelusions or reeommendations 
expressed in this material are those of the author(s) and do not neeessarily refieet the 
views of the National Scienee Foundation. 

References 

Apple, M. W. (2004). Ideology and curriculum. New York: Routledge Falmer. 

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. 
Psychological Bulletin, 84(2), 191-215. 

Betz, N. E. & Hackett, G. (1983). The relationship of mathematics self-efficacy 

expectations to the selection of science-based college majors. Journal of Vocational 
Behavior, 23(3), 329-345. 

Blustein, D. L. (1989). The role of goal instability and career self-efficacy in the career 
exploration process. Journal of Vocational Behavior, 35(2), 194-203. 

Brown, C., Darden, E. E., Shelton, M. E., & Dipoto, M. C. (1999). Career exploration 
and self-efficacy of high school students: Are there urban/suburban differences? 
Journal of Career Assessment, 7(3), 221-221 . 

Darling-Hammond, E (1997). The right to learn. San Erancisco, CA: Jossey-Bass. 

Denbo, S. J. & Beaulieu, E. M. (2002). Improving schools for African American students: 
A reader for educational leaders. Springfield, IE: Charles C. Thomas. 

Dewey, J. (1916). Democracy and education. New York: Tree Press. 


106 



Downing, L. K. (1935). The Negro in the professions of engineering and architecture. 
Journal of Negro Education, 4(1), 60-70. 

Du Bois, W. E. B.(2007). The souls of black folks. New York: Oxford University Press. 

Fouad, N. A., Smith, P. L., & Enchos, E. (1997). Reliability and validity evidence for the 
middle school self-efficacy scale. Measurement & Evaluation in Counseling & 
Development, 30(\), 15p. 

Fouad, N. A. & Smith, P. E. (1996). A test of social cognitive model for middle school 
students: Math and science. Journal of Counseling Psychology, 43{3), 338-346. 

Ford, D. Y., & Harris, J. J. 11 (1996). Perceptions and attitudes of black students toward 
school, achievement, and other educational variables. Child Development, 67(3), 
1141-1152. 

Freeman, D. J., Brookhart, S. M., & Eoadman, W. E. (1999). Realities of teaching in 
racially/ethnically diverse schools: Feedback from entry-level teachers. Urban 
Education, 34(1), 89-114. 

Gordon, H. R. D. (2008). The history and growth of career and technical education in 
America. 3’^‘^ Edition. Fong Grove, IL: Waveland Press, Inc. 

Grayson, E. P. (1980). A brief history of engineering education in the United States. 

IEEE Transactions on Aerospace and Electronic Systems, 16(3), 373-392. 

Gushue, G. V. (2006). The relationship of ethnic identity, career decision-making self- 
efficacy and outcome expectations among Eatino/a high school students. Journal of 
Vocational Behavior, 68(\), 85-95. 

Hargrove, B. K., Creagh, M. G., & Burgess, B. E. (2002) Family interaction patterns as 
predictors of vocational identity and career decision-making self-efficacy. Journal of 
Vocational Behavior, 61, 185-201. 

Kenny, M. E., Gualdron, E., Scanlon, D., Sparks, E., Blustien, D. E., & Jernigan, M. 
(2007). Urban adolescents’ constructions of supports and barriers to educational and 
career attainment. Journal of Counseling Psychology, 54(3), 336-343. 

Eent, R. W., Brown, S. D., & Earkin, K. C. (1986). Self-efficacy in the prediction of 
academic performance and perceived career options. Journal of Counseling 
Psychology, 33(3), 265-269. 

Eeslie, E., McClure, G. T., & Oaxaca, R. E. (1998). Women and minorities in science and 
engineering: A life sequence analysis. The Journal of Higher Education, <59(3), 239- 
276. 


107 



Longstreet, W. S. (1978). Aspects of ethnicity: Understanding differences in pluralistic 
classroom. New York, NY; Teachers College Press. 

Luzzo, D. A. (1993). Reliability and validity testing of the career decision-making self- 
efficacy scale. Measurement and Evaluation in Counseling and Development, 26(2), 
137-142. 

McDermott, R., & Raley, J. D. (2007). From John Dewey to an anthropolgy of education. 
Teachers College Record , 1820-1835. 

Moody, F. (1980). The history of Blacks in vocational education. Journal ofVocEd, 
55(1), 30-34. 

National Science Foundation, Division of Science Resources Statistics. 2006. S&E 

Degrees, by Race/Ethnicity of Recipients: 1995-2004. NSF 07-308. Susan T. Hill and 
Maurya M. Green, project officers. Arlington, VA. 

Navarro, R. L., Flores, L. Y., & Worthington, R. L. (2007). Mexican American middle 
school students’ goal intentions in mathematics and science: A test of social cognitive 
career theory. Journal of Counseling Psychology, 54(3), 320-335. 

Norman, O., Ault, C. R. Jr., Bentz, B., & Meskimen,L. (2001). The Black- White 
“Achievement Gap” as a perennial challenge of urban science education: A 
sociocultural and historical overview with implications for research and practice. 
Journal of Research in Science Teaching, 55(10), 1101-1 1 14. 

Ogbu, J. U. (1987). Variablity in minority school performance: A problem in search of an 
explanation. Anthropology & Education Quarterly, 18(A), 312-334. 

Osipow, S. H. (1983). Theories of career development. Englewood Cliffs, NJ: Prentice- 
Hall. 

Sauermann, H. (2005). Vocational choice: A decision making perspective. Journal of 
Vocational Behavior, 66(2), 273-303. 

Shaffer, S., Ortman, P. E., & Denbo. S. J. (2002). The effects of racism, socioeconomic 
class and gender on the academic achievement of African American students. In S. J. 

S. J. Denbo & E. M. Beaulieu (Eds.), Improving schools for African American students: 

A reader for educational leaders (pp. 19-29). Springfield, IE: Charles C. Thomas. 

Taylor, K. M & Betz, N. E. (1983). Applications of self-efficacy theory to the 

understanding and treatment of career indecision. Journal of Vocational Behavior, 
22(1), 63-81. 


108 



Taylor, K. M., & Pompa, J. (1990). An examination of the relationships among eareer 
deeision-making self-efficacy, career salience, locus of control, and vocational 
indecision. Journal of Vocational Behavior, 37(1), 17-31. 

Woodson, C. G. (1990). The mis-education of the Negro. Trenton, NJ: African World 
Press, Inc. 

Wharton, D. E. (1992). A struggle worthy of note: The engineering and technological 
education of Black Americans. Westport, CN; Greenwood Press. 


109 



IMPACT OF MENTORSHIP PROGRAMS ON AFRICAN-AMERICAN HIGH 
SCHOOL STUDENTS’ PERCEPTIONS OF ENGINEERING 


Cameron D. Denson 
University of Georgia 


Abstract 

The following study seeks to measure the effeetiveness of mentorship programs to 
influenee Afriean-Ameriean male high sehool students’ pereeptions toward engineering 
after partieipating in a eollege-based mentorship program with mentors who are aetive 
members in the National Soeiety of Blaek Engineers (NSBE). In this study, indicators of 
students’ perceptions included students’ career awareness of engineering and self- 
efficacy in the area of math and science. This study used a two-group, posttest only, 
quasi-experimental design. Eollowing participation InNSBE’s mentoring program, the 
treatment for this study, a survey was used to collect data to answer the following 
research questions: 

Research Questions 

1 . Is there a significant difference in career awareness of engineering disciplines 
for students who participated in NSBE’s mentorship program when compared 
with non-mentored students? 

2. Is there a significant difference in self-efficacy in the area of math for students 
who participated in NSBE’s mentorship when compared with non-mentored 
students? 

3. Is there a significant difference in self-efficacy in the area of science for 
students who participated in NSBE’s mentorship when compared with non- 
mentored students? 

Introduction 

The proposed study sought to measure the effectiveness of the mentorship program 
currently utilized by the members of the National Society for Black Engineers (NSBE) 
when matched with African-American male students from the Middle College at North 
Carolina Agricultural and Technical State University. The researcher chose to focus on 
the social interactions of mentorship programs and their potential to influence 
participants’ perceptions. Although previous studies have documented the effectiveness 
of mentorship programs, a recent literature review suggested that there is a lack of 
literature on mentoring that is based on experimental designs (Underhill, 2005). Using a 
posttest-only control group design, this research provided a comparative study on the 
effectiveness of mentorship programs for underrepresented populations. 
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Research Design 

This quasi-experiment utilized a two-group, posttest only design, whieh framed the 
research (Campbell & Stanley, 1997). This research design is useful in studies where the 
administration of a pretest may influence the participants’ behavior during the experiment 
or on the posttest (Gall, Gall, & Borg, 1996). The effects of the treatment administered 
can be measured by comparing the posttest scores of two populations. This research 
design is appropriate when trying to influence a stable characteristic such as students’ 
perceptions towards engineering. 

The dependent variables were identified as students’ perceptions, which included 
students’ career awareness of engineering disciplines and self-efficacy in the area of math 
and science. The mentorship program in which the students participated represented the 
treatment for this study. Mentorship in this study has been defined as “a structured 
mentoring relationship. . .with the primary purpose of systematically developing the skills 
and leadership abilities of less-experienced members of an organization” (Murray & 
Owen, 1991, p. 5). This research study has been carefully designed and should yield 
useful information that can be generalized within margins to the target population of male 
high school students attending the Middle College at North Carolina Agricultural and 
Technical State University (North Carolina A&T, hereafter). 

Random assignment was used in this study to select participants, thus, allowing all 
African-American male students attending the Middle College at North Carolina A&T an 
equal opportunity to be selected for the study. Participants completed an evaluation of 
mentorship program at the conclusion of the study in an effort to monitor and better 
facilitate the mentorship program. Data analysis consisted of performing analysis of 
variance (ANOVA) tests on the acquired means through independent sample t-tests. 
Analysis of variance is highly effective when a researcher is interested in comparing and 
contrasting the mean of two or more samples without increasing the chances of having a 
Type I error. ANOVA allows the researcher to see if differences in the variable of interest 
between groups are due to chance, or if there is a significant group effect. It reports very 
little else about the nature of that relationship, however, it does reveal whether a 
significant difference exists between groups. 

The survey consisted of 3 1 closed-ended questions using a 4-point Likert-type scale 
response whose range consisted of: Strongly disagree=l, Disagree=2, Agree=3, Strongly 
agree= 4. The survey was designed in an effort to gain information about students’ 
perception toward the technical field of engineering. Perceptions included students’ 
career awareness of engineering disciplines and self-efficacy in the area of math and 
science. Participants were not asked to put their name on the survey, thus, protecting their 
anonymity. At the time of the test, participants were notified of their rights of anonymity. 
Demographic information of the participants was collected at the end of the survey 
during the evaluation phase, only identifying the participant’s age (at last birthday), grade 
level and respective mentor. This descriptive data will help aid in forming group 
categories for data analysis. 
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Results - Demographics 

Approximately 24 male students (N=24) out of the 83 Middle College students were 
randomly selected to participate in the study. The treatment group (n=12) consisted of 
students who participated in the NCETE/NSBE mentorship program, and the control 
group (n=12) consisted of students attending the Middle College who did not participate 
in the mentorship program. The sample consisted of predominately Black/ African- 
American (95.8%) and male (100%) students. The grade level break down is as follows: 

8 students, or roughly one -third of the participants were freshman (33.3%), 6 participants 
were sophomores (25%), 4 participants were juniors (16%), and another 6 participants 
were seniors (25%). 

Of the 24 students who participated in the study, 21 surveys yielded useable data. One 
student was considered an outlier due to the fact that his ethnicity was determined to be 
White or Caucasian. Another student did not complete the survey, bringing the total 
number to 22. Upon further analysis, one participant’s responses were deemed invalid 
and unreliable. The markings on the paper and pencil test clearly demonstrated that the 
participant did not complete the survey to the best of his knowledge, which posed a 
problem for validity of the results. With 21 valid entries to compare, the researcher 
randomly eliminated one participant to ensure an even number of participants for the 
control and experiment groups. The total number of useable data resulted in N=20. 

1. Is there a significant difference in career awareness of engineering 
disciplines for students who participated in NSBE’s mentorship program when 
compared with non-mentored students? 

An independent sample t-test was used to compare the means for each construct and 
determine differences that were statistically significant. Eor awareness of engineering 
principles, the mean score for the experimental group was 40.30 and 38.40 for the control 
group. Standard deviations were 5.72 for the control group and 3.95 for the experimental 
group. Although the experimental group produced a higher mean score than the control 
group, these results were not statistically significant at an alpha level of .05 (p=.236). 

2. Is there a significant difference in self-efficacy in the area of math for 
students who participated in NSBE’s mentorship when compared with non- 
mentored students? 

Using the same analysis techniques as described above, results are provided for 
participants’ self-efficacy in the area of math as it relates to engineering. Eor self-efficacy 
in math, the control group yielded a mean score of 23.3 and 22.6 for the experimental 
group. The standard deviation for the self-efficacy in math was 3.75 and 3.62, 
respectively. Though there is a slight difference in the mean scores of the control and 
experimental group, these results failed to reach significance (p=.931). 
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3. Is there a significant difference in self-efficacy in the area of science for 
students who participated in NSBE’s mentorship when compared with non- 
mentored students? 

In a comparison of mean scores for students’ self-efficacy in science as it relates to 
engineering, an independent sample t-test yielded the following results. The experimental 
group produeed a mean score of 28.10, and the eontrol group produced a mean seore of 
25.80. The standard deviation for each group was 4.12 and 3.96, respectively. The 
experimental group produced a mean seore more than two points higher than the control 
group, however, further analysis determined that this researeh question did not produce a 
difference that was determined to be statistieally significant (p=.924). 

Conclusions and Implications 

Past studies have failed in their evaluation of formal mentorship programs, whieh is 
evident by the lack of comparative studies that look at mentorship programs. The 
following study was instrumental in providing a blueprint for evaluating the impact of 
formal mentorship programs to influence the self-efficacy of students. By providing 
quantitative and qualitative data analysis, this study has the potential to provide valuable 
data for researchers looking to increase the retention and recruitment of underrepresented 
populations in engineering fields through mentorship programs. Although the survey 
failed to reveal a difference in mean seore that was statistieally signifieant, no researeher 
of note has attempted to eompare the self-efficacy of students participating in a formal 
mentorship program against those not participating. Further analysis of the quantitative 
and qualitative data should provide answers to the lack of statistical significance. These 
results will be forthcoming. 
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Appendix A 


Perception and Self-efficacy Survey 

Directions: Please write the requested information in the spaee provided or circle the 
number that best refleets your answer to the question. There are no right or wrong 
answers to these questions. We appreeiate your assistanee. 

I. BACKGROUND INFORMATION 

1 . Home Room Teaeher: 

2. High Sehool: 

3. Grade Level; 

4. Gender: 0 = Female 1 = Male 

5. Raee/Ethnieity with whieh you most elosely identify: 

1 = Black/African American 4 = American Indian/Alaskan Native 

2 = Hispanic/Latino American 5 = White/Caucasian 

3 = Asian/Pacific Islander 6 = Other: 

6. What is the highest level of formal edueation completed by your parents? 

Mother Father Mother Father 

Grammar school or less 1 1 College Degree 1 1 

Some high school 2 2 Some graduate school 2 2 

High school graduate 3 3 Master's degree 3 3 

Some college/assoc, degree 4 4 Doctorate/professional degree 4 4 

7. Highest degree expected in your lifetime: 

1 = Associate/technical (2 year degree) 2 = Bachelors (4 year degree) 3 = 
Masters 3 = Doctorate 

8. Approximately how many hours per week are you employed: 

(a) Off-campus: hours/week 

GPA: 

In high school: . 

No. of courses successfully completed to date in: 

Engineering Math Science 

9. Did you: 

1 = enter the school year at the Middle College 

2 = transfer from another high school 
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II. Note: If you participated in the NCETE/NSBE mentorship program please fill 
out section. All others please SKIP section. This section asks about the 
characteristics of the mentorship program and the kinds of activities that go on in it. 
Using the scale helow, please circle the number that best reflects how often you have 
experienced the following in the mentorship program 

I = Never 2 = Oeeasionally 3 = Often 4 = Very Often/Almost Always 
n/a = Not Applieable 

In this course: 


a. Assignments, presentations, and learning activities are clearly 

related to one another. 1 2 

b. I work cooperatively with other students on design challenges 1 2 

c. The team teaches, and learns from each other. 1 2 

d. There are opportunities to work in groups. 1 2 

e. I am encouraged to show how particular knowledge can be 

applied to “real-world” problem. 1 2 

f I have opportunities to practice the skills I am learning in the 
mentorship program. 1 2 

g. I discuss ideas with my classmates (either individuals or in a 

group). 1 2 

h. I get feedback on my work or ideas from my mentor. 1 2 

i. We do things that require us to be active participants in the 

mentoring process. 1 2 

j. The mentor makes clear what is expected of students regarding 

activities and effort. 1 2 

k. The mentor gives me frequent feedback on my work. 1 2 

l. The mentor gives me detailed feedback on my work. 1 2 


3 4 
3 4 
3 4 
3 4 

3 4 

3 4 

3 4 
3 4 

3 4 

3 4 
3 4 
3 4 


n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 
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III. Perceptions of Engineering 

This section asks about conceptual knowledge of engineering as a field and career. 
Using the scale below, please circle the number that best reflects your extent of 
knowledge 1= Disagree 2 = Slightly Agree 3 = Agree 4 = Strongly Agree 

I feel confident in my: 


a. Understanding of what engineers do in industry as 
professionals 


b. Understanding of engineering as a field whieh often calls for 

non-technical considerations (e.g., economic, political, ethical, 12 3 4 

and/or social issues). 

c. Knowledge and understanding of the engineering graphics ^ 2 3 4 

engineering. 

d. Knowledge and understanding of the process of design in 12 3 4 

engineering. 

I feel confident in my ability to: 


e. Do design. 


2 3 4 


f Solve an ill-defined problem (that is, one that is not clearly 
defined). 

g. Identify the knowledge, resources, and people needed to solve 
an ill-defined problem. 

h. Evaluate arguments and evidence so that the strengths and 
weaknesses of competing alternatives can be judged. 

i. Apply an abstract concept or idea to a real problem or 
situation. 


2 3 
2 3 
2 3 
2 3 


4 

4 

4 

4 


j. Divide ill-defined problems into manageable components. 12 3 

k. Clearly describe a problem orally. 12 3 

l. Clearly describe a problem in writing. 12 3 

m. Develop several methods that might be used to solve an ill- 12 3 

defined problem. 

n. Identify the tasks needed to solve an ill-defined 12 3 

o. Visualize what the product of a project would look like. 12 3 

p. Weigh the pros and cons of possible solutions to a problem. 12 3 

q. Figure out what changes are needed in prototypes so that the ^ 2 3 

final engineering project meets design specifications. 


4 

4 

4 

4 

4 

4 

4 

4 
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IV. Self Efficacy in Math 


This section asks about student’s confidence and self belief to use math to solve 
technological and engineering problems. Using the scale below, please circle the 
number that best reflects your perceived ability 1= Disagree 2 = Slightly Agree 3 
Agree 4 = Strongly Agree 


I feel confident in my: 


a ability to accurately calculate numerical 


problems mentally. 

1 

2 

3 

4 

n/a 

b. ability to accurately calculate numerical 





n/a 

problems on paper. 

1 

2 

3 

4 


c. ability to estimate and make approximations. 

1 

2 

3 

4 

n/a 

d. ability to interpret the accuracy of results and 





n/a 

measurements. 

1 

2 

3 

4 


e. ability to calculate the effects of change in 





n/a 

variables using mathematical models. 

1 

2 

3 

4 


f. ability to predict the rate of change of variables 





n/a 

using mathematical models. 

1 

2 

3 

4 


g. ability to use the knowledge and skills in 






mathematics to interpret presentations of 






mathematics 

1 

2 

3 

4 

n/a 

h. ability to learn the material taught in your math 






courses. 

1 

2 

3 

4 

n/a 



IV. Self Efficacy in Science 

This section asks about student’s confidence and self belief to use their 
understanding of science to solve technological and engineering problems. Using the 
scale below, please circle the number that best reflects your perceived ability 1= 

None 2 = Slight 3 = Moderate 4 = A Great Deal 

I feel confident in my: 


a ability to understand the laws of seienee and 

nature to solve problems. 1 2 3 

b. ability to understand natural systems. 1 2 3 

c. ability to understand basie eoncepts of seienee 

and teehnology. 1 2 3 

d. ability to use logieal and systematie thinking in 

scientihe eontexts. 1 2 3 

e. ability to use seienee to solve teehnologieal 

problems. 1 2 3 

f. ability to prediet the rate of change of variables 

using scientific equations. 1 2 3 

g. ability to use science to solve ill-defined 

problems? 1 2 3 


4 n/a 
4 n/a 
n/a 
4 

4 ifa 
4 ifa 
4 ifa 
4 n/a 


h. ability to be part of a problem solving team, 

expressing your ideas, listening and responding 

to others. 1 2 3 4 n/a 

j. ability to learn the material taught in your 

science courses. 1 2 3 4 n/a 


THANKS VERY MUCH FOR YOUR HEUP! 

Please return completed questionnaires to whomever distributed them to the class. 
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Introduction 

In 1983, A Nation at Risk (National Commission on Excellence in Education [NCEE], 
1983) established the resurgence for the scienee, teehnology, engineering, and 
mathematics (STEM) movement in education. In the 1980s, a gradual change in 
economie strength was oecurring, transferring power from domestic industries to foreign 
markets. Though this trend’s history ean be traced back 20 years, it was the A Nation at 
Risk (NCEE) report that brought the reality to the public’s attention. 

The time is long past when Ameriean's destiny was assured simply by an 
abundance of natural resourees and inexhaustible human enthusiasm, and by our 
relative isolation from the malignant problems of older civilizations. The world is 
indeed one global village. We live among determined, well-educated, and 
strongly motivated eompetitors. We compete with them for international standing 
and markets, not only with products but also with the ideas of our laboratories and 
neighborhood workshops. America's position in the world may once have been 
reasonably secure with only a few exeeptionahy well-trained men and women. It 
is no longer, (p. 10) 

The influence of this report and its recommendations are eehoed in the feverish 
development of national standards produeed by academic organizations such as the 
National Council of Teachers of Mathematics (NCTM), the American Association for the 
Advancement of Scienee (AAAS), and the International Teehnology Education 
Association (ITEA). 

Each educational organization was in the proeess of producing preliminary doeuments by 
the close of the 1980s. It is within this proeess that we may witness the first inclinations 
of STEM. NCTM (2000), AAAS (1989), and ITEA (2000) documents ah suggest the 
eombination or integration of their respeetive subjects in an attempt to enhance student 
learning and STEM preparation. In the NCTM’s Principals and Standards for School 
Mathematics (2000), students are encouraged to “pursue an edueational path that will 
prepare them for lifelong work as mathematicians, statisticians, engineers, and scientists” 
(p. 4). The AAAS doeument. Science for All Americans, defines a (seientificahy) literate 
person as “one who is aware that science, mathematies, and teehnology are 
interdependent human enterprises with strengths and limitations” (p. 4). 

ITEA rounds out this example in the following exeerpt from the Standards for 
Technological Literacy; “Teehnology is not simply one more field of study seeking 
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admission to an already crowded curriculum ... it reinforces and complements the 
material that students learn in other classes” (p. 6). 

This proposed subject integration has taken many forms since the overall arrival of 
standards. Programs, modules, packaged curriculums, and even charter schools have 
aligned themselves with proposed models of what a STEM educational program should 
represent. A recent report by the Academic Competitiveness Council (ACC) indicates 
that there are up to 105 government-funded STEM education programs in the United 
States, ranging from kindergarten to post-graduate education. Included in this estimate 
are outreach programs, such as weekend, after school, and summer programs. The report 
by the ACC also collected information regarding the cost associated with STEM 
education programs. Overall estimates indicate total government expenditure to exceed 
$3.12 billion over the 2006 fiscal year. These monies are supplied through a variety of 
government foundations — predominately by the National Science Eoundation (NSE) at 
29%, the Department of Health and Human Services (HHS) at 27%, and the Department 
of Education (DoE) at 23%. 

Evaluations of these programs were collected and examined (ACC, 2007). The 
evaluations were referenced to review the proposed effectiveness of the STEM programs 
and the quality of the evaluations themselves. Unfortunately, it was found that a great 
deal of the evaluations were below the expectations of the council. In fact, those that did 
display potential still required revisions to add greater validity to the information 
provided. This is not a new occurrence. NSE has been revising its own grant procedures 
to account for this lack of efficient evaluation. Educational research grants such as 
Innovative Technology Experience for Students and Teachers (ITEST), Research and 
Evaluation on Education in Science and Engineering (REESE), Discovery Research, K - 
12, (DR-K12), and Informal Science Education (ISE) are now under review in the 
attempt to develop a more in-depth evaluation format than previous documentation (NSE, 
2008). Programs funded by NSE and other organizations have carried on for years with 
government money without providing enough sufficient information or measurable 
influence upon the educational community (ACC, 2007). 

Add to this condition the limitless number of private industries that have produced and 
sold STEM educational products and curricula over the last 20 years. These varied items 
align themselves with national standards and suggest educational advancement in the 
form of problem solving, cooperative learning, and subject integration. Some of the more 
popular examples associated with implementing principles of STEM are Project Eead the 
Way (PETW), DesignQuest (PTC), and Engineering byDesign (EbD). However, very 
little research has been conducted regarding the degree of influence upon education or 
even student learning (Bottoms, G., & Anthony, K., 2005, PTC, 2006, ITEA, 2008). This 
is not to suggest that any of the programs or institutions engaged in STEM endeavors are 
faulty simply that they require more reinforcement and refinement through proper 
research and evaluation. A more recent development is the creation of entire educational 
institutions devoted to STEM development. These schools are not vocational or career 
and technical schools, but rather college preparatory programs designed to develop 
student abilities and interest in STEM and STEM careers. 
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The Problem 


STEM-based educational programs and institutions have been developed to address the 
national need for engineers, technicians, and scientists. Several national organizations 
have displayed great interest in such development. In 2004, the Education Commission of 
the States (ECS) issued the report No Time to Waste: The Vital Role of College and 
University Leaders in Improving Science and Mathematics Education. According to the 
report, the American STEM workforce has quadrupled over the last 20 years. Conversely, 
the number of students preparing for careers in STEM has been either stagnant or 
declining. An interesting correlation is noted in the ECS report to address this condition: 

[CJlassroom access to computers and the internet had expanded significantly, as 
has the availability of Advanced Placement science and mathematics courses. 
Nearly all states have established academic standards in both science and 
mathematics, and the annual testing of students in core subjects mandated by the 
No Child Eeft Behind Act will be extended, in the 2007 - 08 school year, to 
include science. Still, on a number of key indicators, America’s systems of 
science and mathematics education continues to perform below par (p. 3) 

The recommendations from this report consisted primarily of increased science and 
mathematics abilities for future teachers as well as greater exposure and collaboration 
with industry (ECS, 2004). 

In 2005 the report. Tapping America ’s Potential, produced a summary of the concerns 
from a variety of local professional organizations: American Electronics Association, 
Business-Higher Education Eorum, Business Roundtable, National Defense Industrial 
Association, TechNet, and several more. The report cited warnings in the form of a 
declining STEM equipped population: increased foreign competition, low student interest 
toward engineering, low student achievement, and declining research funding (Business 
Roundtable, 2005). This collection of warnings from the Business Roundtable reinforces 
the growing amount of concern within American industry. The American Electronics 
Association (AeA) also shared their concern in the following statement in 2005: 

“America needs to recognize that future innovation is not predetermined to occur in the 
United States. Even if we were doing everything right, we still face unprecedented 
competition from abroad” (p. 3). 

Many of the same organizations have invested in educational programs and institutions 
based upon principles of STEM in hopes of reversing these common concerns. 

It is, therefore, imperative to know if STEM-based schools are actually developing 
student interest in STEM and STEM careers. Currently, no instrument is available to 
measure accurately student interest in STEM. Data provided by such an instrument could 
inform STEM-based programs as to the value of their programs regarding the projected 
influence upon improving the STEM workforce at large. 
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Additionally, the supporting agencies of such educational programs, like STEM-based 
schools, would be clued in to the worth and necessity of their current investments, 
thereby, better determining educational investments based upon research and not hopeful 
outcomes. Large amounts of money and time have already been provided in the hope that 
these institutions will provide student interest and abilities related to STEM. However, 
these donations have yielded little results as demonstrated by the continued reports being 
constructed each year demanding greater STEM investment and results. 

The development of an instrument that can accurately measure student interest in STEM 
careers is crucial to STEM-based programs, their intended outcomes, and the companies 
that aid in their function as well as hope to reap from their products. 

Objectives of the Proposed Study 

The principal objectives of this study are as follows: 

1 . To review and revise current instruments dedicated or related to assessing student 
interest, career interest, and/or STEM education. 

2. To construct a new instrument created from a selection of existing items and 
modeled from the Concerns Based Adoption Model in an attempt to measure 
student interest in STEM careers. 

3. To validate the instrument through the advisement of a panel of experts in or 
related to STEM. 

4. To pilot test the instrument for reliability through a random selection of available 
students currently enrolled within STEM-based schools. 

Intended Method 

In order to create a student career interest instrument specific to STEM-based high school 
environments, an item pool from existing scales will be collected. Permission will be 
sought from organizations representing such instruments. The purpose of the item pool is 
to provide the researcher with a collection of statements that may represent operational 
definitions of the chosen affective characteristics. Domain sampling will provide the 
collection of statements that align with the content domains of the instrument. The 
intended content domains are as follows: 

1 . Student interest in science 

2. Student interest in technology 

3. Student interest in engineering 

4. Student interest in mathematics 

5. Student interest in STEM 
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The selected and composed items will be submitted to a panel of experts to aid in the 
establishment of content-validity. A panel of ten experts in or related to the field of 
STEM will be collected to review the items intended for use upon this instrument. After 
appropriate revisions, pilot testing will commence. An initial pilot test will be conducted 
using a local STEM-based program. The purpose of this initial review is to establish the 
reliability of the selected items in accurately measuring student interest. A second pilot 
test will be conducted using a larger selection of available STEM-based programs. 
Reliability analyses will be produced for both tests. 
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Abstract 

The natural language of scienee is one of multiple representations - seientists “talk” with 
words, numbers, graphs, sehematies, simulations, movies, and so on. These systems of 
representation evolved over time to serve a need for effeetive oommunieation of scienee 
ideas. Students engaged in science learning activities interact with these conventional 
systems of representation, and these expressive modes can impact how they think about 
science. This research aims to uncover more about how students go about representing 
ideas in science and how their ideas and understanding evolve throughout the process of 
representing across multiple systems. Specifically, we introduce a new tool which allows 
students to make stop-action movies about science content in order to see how these 
representations compare with those constructed in other systems. This paper will present 
a pilot research study on children’s multiple representations of ideas in science. 

Introduction 

Representation is a somewhat ubiquitous term in many fields of social science research, 
with a variety of definitions and usages. Kaput (1985) suggests that representation is an 
“undefined primitive whose meaning unfolds gradually through usage within a particular 
domain of inquiry” (p. 38). The literature suggests a diversity of definitions for 
representation: Enyedy (2005) offers that representation is “the act of highlighting 
aspects of our experience and communicating them to others and ourselves” (p. 427); 
Goldin and Shteingold (2001) suggest that a representation is “typically a sign or a 
configuration of signs, characters, or objects... the important thing is that it can stand for 
(symbolize, depict, encode, or represent) something other than itself’ (p. 3); and Lee and 
Karmiloff-Smith (1996) affirm the notion that representation “establishes a ‘stand for’ 
relationship between referent and sign” (p. 127; see also Kaput, 1991, 1998). These 
definitions highlight important aspects of representation which serve as focal points for 
education research in this area. Symbolization and symbol-use (Nemirovsky, 1994), 
referential-communicative aspects of representation (Tolchinsky, Landsmann & 
Karmiloff-Smith, 1992), and cultural and social aspects of developing conventional 
systems of representation (Confrey, 1991) are all concepts being explored. 

Representations are often considered from two perspectives, internal and external 
(Goldin, 1998; Zhang & Norman, 1994). While some researchers argue that this 
distinction may be spurious (i.e., “The Phantom of Dualism”, see Nemirovsky, in press), 
external representations of knowledge are the only primary sources of student 
understanding that researchers are able to collect. Since scientists seamlessly navigate 
across systems of external representation (such as oral language, written language, 
mathematical notation, graphing, and gesture) in the practice of researching and 


127 



communicating results, research into how students represent scientifie knowledge has the 
potential to further this field of study. Students learning seienee must find meaning in 
partieular systems of representation while also learning how to appropriately employ 
them. Additionally, students likely represent their knowledge differently in different 
systems of representation (Brizuela & Earnest, 2007; Goldin, 1998). A great deal of 
knowledge about how students learn ean be gained through studying how they represent 
their understanding in seienee. 

The aet of representing knowledge helps to strengthen and refine one’s understanding of 
a partieular eoneept (Kaput, 1991; Olson, 1994). This work aims to investigate how 
iteratively representing understanding aeross multiple systems impaets knowledge. 
Enyedy (2005) eoined the term “progressive symbolization” as a eonstruct for deseribing 
that as children are asked to externalize their ideas, the artifacts they produee and their 
understanding evolve over time and approaeh eonventional forms. While eonventionality 
is not of immediate eoneern herein, the notion of representing in different systems holds 
great potential for unveiling how students make sense of seientifie practiees. It is 
reasonable to assume that some systems of representation may be better for representing 
eertain aspeets of seienee (e.g., statie vs. dynamie seenarios). Thus, this work will foeus 
on students’ representing their ideas in a number of ways, and it is driven by three 
essential researeh questions 

Research Questions 

1 . How do students represent their ideas about air as a substance through generating 
animated explanations of observed demonstrations? 

2. How are representations produeed through animations both similar and different 
from representations produeed in other systems such as oral language, writing, 
and drawing? 

3. What differenees, if any, exist in the kinds of eoneeptual aspects about air as a 
substance that ehildren are able to represent through different media? 

This work introduees a new system of representation (assuming that drawing, building, 
writing, and speaking ean be agreed upon as existing systems), stop-action movie 
making, also known as animation ^ . Researeh eoneerning animation in instruetional 
settings has largely foeused on using animations as demonstration tools (Hagerty, 1992; 
Kaiser, Proffitt, Whelan, & Heeht, 1992; Mayer & Anderson, 1991, 1992; Mayer & 
Moreno, 2002). While some report gains in eontent reeall and problem-solving transfer 
for students who view animations on eertain concepts (Hagerty, 1992; Kaiser et ah, 1992; 
Mayer & Anderson, 1991, 1992; Morrison, Tversky, & Betraneourt, 2000), the researeh 
does not present a eonsensus for how animations should be utilized as a learning aide. 

The literature presented does not appear to be in agreement on how animation ean be 
effeetive for use in learning situations. However, the researeh has focused largely on 


* An animation is defined as a short movie (typically less than thirty seconds) comprised of individual still 
images that are played in rapid succession (set by a frame rate), similar to a flip book. 
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animations as demonstrations, rarely as artifaets eonstrueted by students. Air and air 
pressure are ehosen as the eontent foeus beeause they are generative eoneepts (Basea & 
Grotzer, 2001), in that they ean eause visible aetions while remaining “unseen”, making 
them rich contexts for investigating how students represent their ideas. 

Methodology 

To investigate the differential impact of using animations versus other forms of 
representations, the proposed pilot study intends to elicit multiple representations from 
students across multiple systems. These representations will be produced after students 
interact with science demonstrations that are designed to highlight conceptual aspects of 
air and air pressure. Morrison and Tversky (2001) "Conceptual Congruence Hypothesis" 
suggests that since animations depict changes over time, the situations about which 
students are asked to generate animations should also include elements of motion and 
change over time. Thus, students will interact with demonstrations that show a change 
over time, and they will be asked to represent their understanding of these situations in 
multiple ways. The specific exploration involves two plastic syringes connected at the 
nozzles by a short piece of rubber tubing. In this demonstration, as the plunger of one 
syringe is depressed, the plunger of the other syringe extends (Figure 1). This 
demonstration has been used by other researchers in investigations of children's notions 
of air pressure (deBerg, 1995; Sere, 1982; Tytler, 1998) and will be referred to as the 
"Sealed- Syringe" demonstration. Previous research has shown students begin to develop 
ideas about air as a substance as early as age five (Driver et ah, 1994). However, for the 
study proposed here, the subjects will be middle school-aged children (ages 12-14 years), 
as they have been found to have semi-normative ideas about air and pressure (Sere, 

1982). The total number of participants for this study will be 8 students, 4 female and 4 
male, chosen at random from the pool of participants having given informed consent. 
Student-generated animations are the mode of representation under investigation; 
however, these representations will need to be compared with other systems. In addition 
to asking the students to animate their explanations for the observed phenomena (as seen 
through the demonstrations), students will also (A) represent their ideas verbally through 
a semi-structured interview; (B) draw pictures to facilitate an explanation, and (C) use 
physical elements as aids in verbally describing the scenario (such as balloons, fans, clay, 
etc.). All representations will be elicited in the context of a clinical interview, a well 
documented methodology in education research (Brizuela, 1997; Duckworth, 1987, 1996; 
Piaget, 1965). The novel system, animation, will be employed using the Tufts University 
Center for Engineering Educational Outreach SAM Animation software. SAM is an 
environment that allows children to make animations with the aid of computer imaging 
and processing. In SAM Animation, a web-camera is connected to the computer which 
displays a live video image in one window of the software (see Appendix A for a detailed 
description of the software). This allows the user to make the animation out of whatever 
materials they desire (e.g., drawings, manipulatives, paper cut-outs, EEGO bricks, etc.). 


While the term "demonstration" is used here, these activities could be considered "explorations", because the 
student will have the opportunity to interact with the materials in order to gather first hand experience with the 
phenomena. 
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The user "snaps" the image they want, and a still picture is recorded and placed in a 
timeline. The user then adjusts the scene captured by the camera and "snaps" another 
image. The animation is, therefore, a collection of still images taken from the camera 
which is focused on some scene external to the computer. The user can "play" the 
animation at a specific frame rate (measured in frames per second) which they prescribe. 
The result is a computerized movie, which can be exported as a Quicktime™ file and 
shared with other students, teachers, and researchers. Church, Gravel, and Rogers (2007) 
have previously reported success using this software with students in high school physics 
courses. In the proposed research, students will be taught how to use the animation 
software (in a minimum of two sessions) prior to the introduction of the air 
demonstrations. Each participant will take part in three research sessions: (1) a session 
focused on the exploration, a verbal interview, and drawing representations; (2) a session 
where each student is offered a range of materials from which to make animated 
representations of their ideas; and (3) a session where the participant builds a physical 
artifact to aide verbal explanations. All artifacts will be collected and analyzed alongside 
the transcripts of each clinical interview. 

Implications 

This work has the potential to uncover patterns and trends in how students spontaneously 
represent their understanding of scientific ideas across multiple systems. While it is 
important to note that definite lines between each of the proposed systems cannot be 
drawn without generating theoretical concerns (i.e., considering speech as completely 
independent from written notation can be argued against), perhaps, each of these 
environments allow students to represent aspects of their understanding in different ways. 
The introduction of animation as a new representational environment alongside more 
traditional systems highlights the aspects of representation in science that need further 
exploration in a dissertation study. 
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Figures 



Figure 1. The "Sealed-Syringe" demonstration device. A fixed amount of air is enclosed 
in the system such that depressing one plunger results in opposite plunger extending. 
Students will also be asked to press both plungers down simultaneously, compressing the 
air inside the system which is another conceptual aspect of the interviews. 
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Appendix A 

SAM Animation is a free-of-charge computer software program written at the Tufts 
University Center for Engineering Educational Outreach. Eigure 1 shows the main screen 
of SAM Animation and a description of relevant features follows. 



Figure 1. SAM Animation software. 

The features of the software are labeled and are described below: 

A. Main animation screen -in this window, the most recent still image taken from the 
camera is displayed. Pressing the "play" button beneath the window will play the 
animation at the prescribed frame rate (set by the slider beneath the window and "play" 
button) 

B. Camera view -this window shows the image being fed in from the webcam connected 
to the computer. Beneath this image is a button with the image of a camera on it. Clicking 
here captures a still image from the video feed and adds that image to the timeline at the 
bottom of the screen. 

C. Onion-skinning -this button allows the user to see a faded image of the last still image 
captured. This helps the user in aligning scenes created for the animation. 

D. Audio function -by clicking this tab, the audio screen is accessed. Here the user can 
add one audio track to the animation. 
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Abstract 

This study sought to understand the cognitive processes and procedures employed by an 
expert and a novice engineer in a realistic radio frequency systems engineering design 
challenge by using verbal protocol analysis. The engineering design challenge 
encompassed engineering, political, and social constraints. The audio data was then 
transcribed, segmented, and coded for analysis. 

The processes and strategies of the expert and novice were juxtaposed for analysis. The 
expert and novice shared some similarities in their cognitive processes and strategies. 
However, the expert’s domain knowledge and experience was vastly distinct from that of 
the novice. Finally, a few ideas on improving engineering and technology education are 
presented. 

The demand for engineers continues to grow while the supply of domestic engineers 
entering the work force continues to decline (NAE, 2004). Undoubtedly, an improvement 
in engineering education curriculum and pedagogy would mitigate this decline. Yet, 
research in engineering education and cognition is minimal in the United States. 
Therefore, more research is needed to further understand engineering education and its 
role in the complete engineering enterprise. This study aims to enhance the engineering 
cognition knowledge base by analyzing and comparing the cognitive processes and 
strategies of an expert and novice in systems engineering design. 

Research Questions 

1 . What cognitive processes and strategies are used by expert and novice engineers 
in systems design? 

2. How do the expert and novice cognitive processes and strategies compare? 

Background 

This study is based on the foundation of cognitive science as it pertains to engineering 
and technology education (Brown, 2001). The novice is limited by experience and 
knowledge resulting in a partial and simple schema. The expert has a vast depth of 
experience and focused practice within a domain resulting in deep and rich schemata 
(Cross, 2004). Experience and knowledge do not ensure expertise. The manner in which 
the experience and knowledge is ordered and interrelated has a great impact on expertise. 
An expert is able to recognize large amounts of information, or chunks (Egan & 
Schwartz, 1979). Erom these chunks, an expert can recognize what information is 
relevant to the issue at hand. This enables the expert to wade quickly and efficiently 
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through data and facts with fast retrieval from his/her memory or schemata. This 
deliberate and focused effort was explained by Eriesson (2001) as the primary differenee 
between experts and those who are only profieient in their domains. 

Design is a nebulous proeess that may be pereeived from either a scientifie or an artistic 
viewpoint (Cross, 2001). Design is also dynamie and iterative, therefore, it is not easily 
represented by simplistie linear models (Mawson, 2003). Design, typieally, eommenees 
with defining the problem space (Cross, 2004). As a result of their laek of design skills, 
noviees spend a large proportion of their time in the problem space (Cross, Christiaans & 
Dorst, 1994). 

With a vast schema based on multiple experienees and focused praetiee, an expert is able 
to identify quiekly pertinent data in the problem space. As experts move quiekly through 
the problem spaee, they plaee a majority of their efforts in the solution spaee generating 
one or multiple design artifaets (Lawson & Dorst, 2005). However, as the solution spaee 
evolves and elucidates further eonstraints, the expert returns to and referenees, or 
redefines, the problem spaee iteratively until the design is implemented, tested, and 
eoneluded. Additionally, experts design from a solid groundwork of “first prineiples”, or 
seientifie theory on whieh to base their design eoneepts (Cross, 2002). These attributes 
eombined give “know how” that is often demonstrated by an expert. Furthermore, 
Ericsson (2001) states that affeetive attributes, sueh as foeus and eommitment, are a 
factor in expertise. 

Methods 

The sample for this study ineluded two partieipants drawn from opposing ends of the 
expertise continuum in the domain of radio frequency (RE) engineering system design. 

As sueh, they were seleeted on their skill set within RE engineering system design. 
Expertise in RE engineering is generally obtained through extensive praetiee in industry 
due to the frequent eomplex human interaetions that must be balaneed with sound 
engineering design. 

The expert for this study has over thirteen years of RE systems engineering design 
working for T-mobile in various positions ranging from manager to internal eonsultant. 
The expert reeeived formal training with a baehelors degree in electrical engineering. The 
other participant is on the other end of the speetrum of RE systems engineering, 
somewhere between noviee and an advaneed beginner. He is a professor in eleetronie 
engineering teehnology and has been teaehing eleetronies at the post seeondary level for 
more than 35 years. Although he is a noviee in RE systems engineering design, he has a 
breadth of skills in pedagogy and undergraduate eleetronies. 

The partieipants were asked to design a new wireless eellular network in an isolated 
eollege town as though they were engineering design eonsultants. Constraints were 
placed in the design ehallenge to ereate a realistie, ill-defined seenario. The eonstraints 
were to limit eapital expenditures and abide by the zoning not to exeeed 60 foot towers, 
and design eell sites to be hidden or stealth. Additionally, the partieipants were made 
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aware of high cellular traffic venues, such as the university with 18,000 students and a 
fictitious annual wakeboarding event that would draw 10,000 individuals. A three 
dimensional aerial map overlaid with major and minor transportation thoroughfares was 
given to the engineers to aid in their design as seen in Appendix A. 

The audio data was broken into units or segments. The segments were then coded into 
distinct mental processes used in engineering design. Although there are various methods 
utilized in coding verbal protocols (Atman & Bursic, 1998; Kruger & Cross, 2001), the 
Halfm (1973) coding method was used in this study as it specifically addressed 
engineering design. These mental processes are presented in Appendix B. 

Results 

As both participants had multiple years of experience at the systems level in electronics, 
each initially utilized a top-down approach in their design. Both participants also used an 
iterative process evaluating and visualizing their design against the various constraints. 
However, the expert was able to analyze more thoroughly and balance the constraints, 
such as zoning and leasing. The expert and the novice each frequently returned to first 
principles for predictions and site locations. However, the novice was fully aware of his 
limitations and stated repeatedly that he did not have the experience and knowledge to 
make an accurate design. Conversely, the expert was able to make mental predictions or 
visualizations of the design and relied heavily on experiential and episodic memory. As 
seen in Appendix C, the percentage of time on task in design was nearly the same at 37%. 

The expert’s design strategy revealed differences from that of the novice. The expert 
approached the design from a personal viewpoint, drawing heavily from previous 
experiences and precedents. One of the most striking contrasts between the participants 
was the attention the expert gave to the optimization of capital expenditures. This same 
theme pervaded the expert’s entire design process. While the expert made sixteen 
references to costs, the novice only mentioned costs in three instances. Additionally, the 
expert’s design proposed only seven sites, the novice fifteen, substantially reducing the 
cost of the proposed design. The differences may be partially attributable to the novice’s 
career in academia while the expert has exclusively been in industry with responsibilities 
as a manager and principal engineer with cost as a consistent constraint. 

Another striking difference between the expert and novice was the amount of domain 
knowledge. Appendix E is a pair of concept maps that reveal the disparity in knowledge. 
The novice clearly did not have the breadth and depth of knowledge as did the expert. 
However, the concept maps reveal that the novice did have a working knowledge of radio 
frequency electromagnetic wave propagation. The novice made mention of zoning, 
leasing, and capacity, but this could partially be accounted for by the design brief 

The expert made use of techniques unique to his trade, or gambits, to help overcome the 
stealth requirements. The expert employed water towers, rooftops, and stadium lights as 
economical alternatives to other costly stealth solutions. When the novice was prompted 
for further analysis and design he replied, “Experience would probably tell a person more 
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information whether [the system design] is enough or. . .not.” Clearly, the noviee had a 
lack of relevant experience and domain specific knowledge to elaborate on his design. 


Discussion 

Engineering and technology education would do well to educate the students broadly to 
become global thinkers (National Academy of Engineering, 2005). This global approach 
to teaching should include costs, organizational behavior, political and societal impacts. 
The design method may be taught, but an emphasis should be placed on the fact that there 
is no universal problem solving model. Einally, systems level engineering could be 
infused into a curriculum similar to that employed by Frank (2005) with a top-down 
approach. Presenting the overall concept and then delving into components is an 
alternative method to accommodate a variety of student learning styles (Felder & 
Silverman, 1988). This paper has presented a few ideas that should be infused into 
engineering and technology education practice and research to increase technological 
literacy. 
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Appendix B 

Halfin’s Mental Processes and Definitions 

Code Mental Process and Definition 

(DF) Defining the Problem or Opportunity Operationally. The process of stating or 

defining a problem that will enhance investigation leading to an optimal solution. It 
is transforming one state of affairs to another desired state. 

(OB) Observing. The process of interacting with the environment through one or more of 
the senses. (Seeing, hearing, touching, smelling, and tasting.) The senses are utilized 
to determine the characteristics of a phenomenon, problem, opportunity, element, 
object, event, system, or point of view. The observer’s experiences, values, and 
associations may influence the results. 

(AN) Analyzing. The process of identifying, isolating, taking apart, breaking down, or 
performing similar actions for the purpose of setting forth or clarifying the basic 
components of a phenomenon, problem, opportunity, object, system, or point of 
view. 

(VI) Visualizing. The process of perceiving a phenomenon, problem, opportunity, 
element, object, event, or system in the form of a mental image based on the 
experience of the perceiver. It includes an exercise of all the senses in establishing a 
valid mental analogy for the phenomena involved in a problem or opportunity. 

(CO) Computing. The process of selecting and applying mathematical symbols, 

operations, and processes to describe, estimate, calculate, quantify, relate, and/or 
evaluate in the real or abstract numerical sense. 

(CM) Communicating. The process of conveying information (or ideas) from one source 

(sender) to another (receiver) through a media using various modes. (The modes may 
be oral, written, picture, symbols, or any combination of these.) 

(ME) Measuring. The process of describing characteristics (by the use of numbers) of a 
phenomenon, opportunity, element, object, event, system, or point of view in terms 
which are transferable. Measurements are made by direct or indirect means, are on 
relative or absolute scales, and are continuous or discontinuous. 

(PR) Predicting. The process of prophesying or foretelling something in advance, 
anticipating the future on the basis of special knowledge. 

(QH) Questioning and Hypothesizing. Questioning is the process of asking, interrogating, 
challenging, or seeking answers related to a phenomenon, problem, opportunity, 
element, object, event, system, or point of view. Hypothesizing is a process of 
stating a theory of tentative relationship between two or more variables to be tested 
which are aspects of a phenomenon, problem, opportunity, element, object, event, 
system, or point of view. 

(ID) Interpreting Data. The process of clarifying, evaluating, explaining, and translating 
to provide (or communicate) the meaning of particular data. 
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Code Mental Process and Definition 


(MP) Constructing Models and Prototypes. The process of forming, making, building, 
fabricating, creating, or combining parts to produce a scale model or prototype. 

(EX) Experimenting. The process of determining the effects of something previously 
untried in order to test the validity of a hypothesis, to demonstrate a known (or 
unknown) truth or try out various factors relating to a particular phenomenon, 
problem, opportunity, element, object, event, system, or point of view. 

(TE) Testing. The process of determining the workability of a model, component, system, 
product, or point of view in a real or simulated environment to obtain information for 
clarifying or modifying design specifications. 

(DE) Designing. The process of conceiving, creating, inventing, contriving, sketching, or 
planning by which some practical end may be effected, or proposing a goal to meet 
the societal needs, desires, problems, or opportunities to do things better. Design a 
cyclic or iterative process of continuous refinement or improvement. 

(MO) Modeling. The process of producing or reducing an act, art, or condition to a 

generalized construct which may be presented graphically in the form of a sketch, 
diagram, or equation; presented physically in the form of a scale model or prototype; 
or described in the form of a written generalization. 

(CR) Creating. The process of combining the basic components or ideas of phenomena, 
objects, events, systems, or points of view in a unique manner which will better 
satisfy a need, either for the individual of for the outside world. 

(MA) Managing. The process of planning, organizing, directing, coordinating, and 
controlling the inputs and outputs of the system. 
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Appendix C 

Percentage of Time on Task 




38% 


□ AN □ CM □ CO □ DE ■ DF □ ID □ MA □ OB ■ PR □ OH □ VI 
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Time on Task (min:sec) 


Appendix D 

Time on Task 
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Appendix E 

Concept Maps of Expert and Novice 
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PERCEPTIONS OF CREATIVITY IN ART, MUSIC, 
AND TECHNOLOGY EDUCATION 

David Strieker 
University of Minnesota 


Abstract 

This study was conducted to examine the perceptions of art, music, and technology 
education teachers with regard to creativity in their respective fields. Participants were 
drawn from the membership databases of the Minnesota Technology Education 
Association (MTEA), Art Educators of Minnesota (AEM), and Minnesota Music 
Educators Association (MMEA). The survey used in this study was designed around the 
themes borne out of ereative literature, generally, and creativity specific to the fields of 
art, music, and technology and engineering education. As a result, the themes of creative 
process, products, personal traits, and environment shaped the items contained in the 
survey. Significant differences were found between the subject areas in all themes 
mentioned above. Specifically, technology educators were less interested in the creative 
process, perceived creative products differently, valued creative personality traits to a 
lesser extent, and viewed the creative environment differently when eompared with their 
peers in art and music. Einally, when compared with other demographic variables, the 
subject (art, music, or technology education) the partieipants taught was the only 
significant determinant of creativity perceptions in the study. 

Introduction 

Business and engineering communities emphasize the importance of ‘outside the box’ 
thinking and the need for creative solutions as a result of competitive market pressures 
that characterize the true global economy that exists today (Mahboub, Portillo, Eiu, & 
Chandraratna, 2004). As a result, a question arises: Where in the curriculum are students 
allowed to exereise their innate creative urges? More specifically, since it is such a 
valued skill, how is creativity fostered in students? Art, music, and technology education 
may be the answer (Eewis, 2008). Eor technology education, specifically, a subject that 
has set a historical precedence for fostering creative work (see Cohen, 1998; Dewey, 
1916; Woodward, 1882, 1883), its current curricular efforts of infusing engineering 
eoneepts that demand creative thinking is of particular interest to educators at all levels 
within the field. Basic to contemporary art education, whose foci are on visual culture 
that emphasizes creative experiences, are these same issues of creative and critical 
thinking skills, as well as problem solving (Ereedman, 2003). The curricular goals of 
general creative and critical thinking along with problem solving, and creating products 
within a certain soeial eonstruct are demonstrated in music education, as well (Webster, 
1987a, 1987b, 1989). Technology education is not the only discipline, therefore, to 
declare that their curriculum champions creativity. Art and music lay claim to many of 
the same types of intellectual rigor in creativity to whieh the engineering-focused 
technology education curriculum seems to assert a monopoly. In light of this, since 
ereativity is a cornerstone of engineering education currently embraced by technology 
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education, it must be determined if the engineering-focused technology education 
curriculum pushes students to explore creativity unique to technology education, but in a 
way that is not attainable in art or music education. 

Purpose of the Study 

Identification of creative aspects inherent to the design and problem solving activities 
being suggested by the new engineering-focused technology education curriculum is a 
fledgling area. The primary purposes of this study, therefore, were to identify specific 
aspects of creativity shared by the subjects of art, music, and technology education and to 
determine if there are creativity aspects unique to technology education. In essence, does 
the technology education curriculum, with its emphasis of engineering and design 
principles, offer students an avenue to explore their creative potential in a way that art 
and music education cannot? As a result, the study sought to answer the following 
questions shaped by literature pertaining to creativity, generally, and within the three 
fields previously mentioned: 

1 . Do technology, art and music teachers differ in their perception of the creative 
process? 

2. Do technology, art and music teachers differ in their perception of the creative 
product? 

3. Do technology, art and music teachers differ in their perception of creative 
personal traits? 

4. Do technology, art and music teachers differ in their perception of the creative 
environment? 

5. Are there predictors of the creativity perceptions among art, music, and 
technology education teachers? 

Methodology 

Identification of attributes inherent in creative work in art, music, and technology 
education is a significant vein that runs through the questions above. Plucker and Runco 
(1998) stated that creativity has lately been considered to be content specific and both 
theoretical and empirical evidence has been provided to make this claim. In addition, 
Baer (1994) said the assessment of creativity should not only be content specific, but 
task-specific within content areas. Therefore, identifying whether a general agreed upon 
group of attributes believed to be common to all creative endeavors was fundamental in 
establishing a starting point for this inquiry. Using these concepts, the development of a 
seventy-nine item survey instrument was developed to collect data from the association 
members mentioned previously. 
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Descriptive statistics including means and standard deviations were used on the data 
gathered from all items, including demographics. Mean comparison and rankings were 
conducted to determine if one subject area perceived certain creative items as more 
important when compared to another subject. In an effort to identify any predictors 
associated with the educators’ creativity perceptions, demographic information was 
collected. MANOVA was used to test these dependent variables as a combined set of 
attributes concerned with each respondent’s total score added across all the items used to 
indicate the level of importance and whether the demographic data interacted differently 
with them. Separate ANOVAs were conducted if significant relationships were found. 

Instrumentation 

The items contained in the survey were sectioned into five categories: one addressing 
demographic information and four dealing with the nature of creativity consistent with 
the literature: creative process; creative product; creative personal traits; and the creative 
environment. The categories and the number of items contained in each of them, seminal 
authors, and common indicating terminology embedded in the literature and, therefore, 
used to compose the items in the survey, are found in Appendix A. Participants rated via 
a seven point Likert-type scale, with 7 indicating “extremely important” and one 
indicating “not important”, the importance of each item relative to their particular field. 
The participants were, also, asked five demographic questions related to the subject and 
grade level they taught, how long they had been teaching their subject, current level of 
education, and gender. The survey instrument is included in full in Appendix B. 

Results 

Although participants from all three subjects perceived the creative process as important 
to creative work, generally, technology education teachers were less interested in the 
importance of the creative process than the teachers of art and music (Appendix C). In 
addition, technology education teachers perceived a product’s ease of use, practical 
implications, value to the community, craftsmanship, ability to respond to a need, and 
general adherence to technical standards as being important features of a creative product 
in their field when compared to art and music teachers (Appendix D). Art teachers valued 
creative personality traits significantly more than their peers in technology education 
(Appendix E). The perception of the importance group work and competition was 
significantly higher for technology teachers than for art teachers (Appendix F). 

Finally, of the variables of subject (art, music, or technology education) taught, grade 
levels taught, years of teaching experience, level of education, and gender, the subject the 
participants taught was the only significant determinant of creativity perceptions in the 
study (Appendix E). 

Conclusion 

Evidence regarding the implementation engineering curriculum has been encouraging. 
For example, Ya§ar, Baker, Robinson-Kurpius, Krause, and Roberts (2006) found 
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teachers were supportive of the idea of infusing design and technology into the 
curriculum. However, Yasar’s et al. research, also, revealed that these teachers had 
negative perceptions of engineers, generally. As demonstrated in the engineering and 
engineering education literature, creative thinking is the foundation to successful design 
within a contemporary technology curriculum. Since the perceptions of technology 
education teachers found in this study were significantly different with regard to items 
focused on a variety of creativity characteristics, this may foreshadow difficulty in the 
full acceptance of the engineering-focused technology education curriculum in the field 
of technology education. The results of this study should be used to initiate a dialog 
regarding the capability of the field of technology education to embrace the types of 
creativity valued by the art and music fields. 
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Appendix A 

Components of the Survey 


Categories 

Seminal Author(s) 

Example Indicators 

Process 
(16 items) 

Lubart(1999, 2000-2001), 
Torranee (1966), 
Sternberg (1985) 

ability to transform things, 
observation, risk taking, 
flexibility, flow, synthesis, 
inventive use of an idea, 
act in a systematic manner 

Product 
(14 items) 

Getzels & Csikszenthmihalyi, 
(1976), 

Eisner (1962) 

originality, completeness, 
novelty, technical quality, 
expressive power, 
aesthetic quality 

Personal Traits 
(29 items) 

Torranee (1963), 
Guilford (1950, 1976), 
Millar (2002) 

humor, playfulness, ability to 
fantasize, ability to delay 
closure, tenacity, sensitivity to 
beauty, awareness of 
feelings/senses 

Environment 
(15 items) 

Amabile, T.M. (1983, 1990). 

time management, sensory 
input, individual work, 
teamwork, knowledge, 
aptitude, technology 
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Appendix B 
Survey Instrument 


Perceptions of Creativity in Art, Music and 
Technoiogy Education 


Consent Form 

You may decline to answer any questions, and you are free to withdraw from the siavey at arry time. 

This research survey is part of a dissertation project and subsequent pubAcations intended to gain irtsight into 
how you feel creativity is approached in your field. Your resportse will be compied for data analysis with the 
goal of assessir>g how Art Technology, and Music teachers view the creative process, creative products, 
creative environment arid personality traits of creative people relative to their subject. All responses of the 682 
participants in this study will be kept confidential and r>o individual participant will be identified in the research 
process. 

There are no foreseeable risks and no direct benefits for your participation in this survey. All of the responses 
wfl be kept confidential arid each potential participant has been assigned a laiique code only for the purposes 
of data analysis arid ekmnating your name from future contacts to complete the survey. The data from the 
survey wil be kept by research staff at the University of Mirwiesota for only research related to the objectives of 
this project 

Again, we want to stress that your identity wil be kept confidential ar>d only results of colective data analysis 
may be used for publication. Your participation is voluntary and r>o loss of benefits or rights will be 
endured if you choose not to participate. 

If you have any questions about this research project, please contact David Strieker at 612-S24<6204 or 
via email stri0026Qumn.edu or you may contact Dr. Theodore Lewis at 612-624-6204 or via email 
lewis007Qumn.edu. If you have questions regarding your rights as a participant, please contact the 
University of Minrtesota Institutional Review Board (I^B) office at 612-626-5654 or irbQumn.edu for 
assistance. 


Directions : Under each of the four sections found below, you wil fnd specific directions that guide you in how to 
complete the survey. Art. Technology, arxl Music teachers will be responding to this survey arxl particular items 
may be more temiliar or relevant to each field. 


J Demographics 

This section of the questionnaire deals with background information relating to you and your school (Mark only 
one response for each item). 


1 . What subject do you currently teach? 

0 Art 

O KAjsic 

O Technology Educatian 

2. What grade level do you currently teach? 
Q Middte/Jiaiior High 

0 High School 
O Other 


4. What is your current level of education? 
O Bachelors degree 

O Masters Degree 
O Doctorate 

5. Gender: 
o Male 

O Female 


How many years have you been teaching your subject? 
0 1-10 
o 11-20 
o 21-30 
0 31 or more 
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This section of the questionnaire asks you to evaluate the extent to which each of 16 
statements relating to processes involved with creative work are consistent with the 
way teachers in your field think atxxjt creativity. Please fii in the number (1 = not 
important, 7 = extremely important) you feel best reflects how essential these 
statements are to the creative process in yoir subject area. 

1 . Having relevant knowledge of prior products or solutions is an important aspect of 
creative work. 

2. To produce creative work a person must be familiar with standards for acceptable 
solutions. 

3. The creative process requires the ability to generate a number of exploratory ideas 
or solutions. 

4. Finding or identifying challenging proWems is a critical dimension of the creative 
process. 

5. Creativity includes the ability to find gaps, inconsistencies or flaws in existing 
solutiotvs. 

6. Generating a representation of the problem or challenge is part of the creative 
process. 

7. Seeking out reactions to possible solutions is an important dimension of the 
creative process. 

8. The creative process sometimes requires taking a break from the problem or 
challenge at hand to allow ideas to incubate. 

9. Creative solutions sometimes come to mind as a “flash" or sudden awareness. 

10. Metaphors and analogies are useful aids in creative thinking. 

11. The act of creating sometimes involves reformulation of the initial problem or 
challenge as one becomes engaged in the work. 

12. The creative process may begin even though the final product may not be formed 
in the "mind's eye”. 

13. The creative process often includes gathering and drawing upon all resources that 
can be helpful in completing a task. 

14. The possession of relevant knowledge is an important aid to the creative process. 

15. Creativity is improved if a person that is familiar with technical rules. 

16. Creativity is improved if a person is familiar with relevant principles or theories. 
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Product 


This section of the questionnaire asks that you to evaluate the extent to which each of 

14 statements relating to the nature of creative products are important in your field. 

Please fill in the number (1 = not important, 7 = extrerrtefy importanf) you best 

reflects how essential statement is to the creative products in your subject area. 

1 . In my field, a creative product must posses a high degree of novelty. 

2. A creative product is likely to influence or suggest additional future creative 
products. 

3. A product is considered creative if it is unusual or seen infrequently in the 
category to which it belongs. 

4. The degree to which a product responds to a need or problem determines its level 
of creativity. 

5. A creative product follows the accepted and understood rules of the discipline. 

6. A creative product has clear and practical implications 

7. To be considered creative, a product in my field must be of value to the community 
at large. 

8. A creative product breaks with the tradKion from which it emerges. 

9. Products are creative if they combine elements in unusual ways. 

10. A product is creative if it commands the attention of a person using, listening to, or 
viewing it 


■■1J®®®®®(-!J 

®®®®®®® 

®®®®®®® 

®® 0 ®®®® 

®®®®®®® 

®®®®®®® 

®®®®®®® 

®®®®®®® 

®®®®®®® 

®®®®®®® 


154 


Product (continued) 


11. A creative product in my field is easy to understand, interpret, or use. 

12. The craft component of completed works is critical in determining how creative 
they are. 

13. To be deemed creative, a prxjduct in my held must be levolubonaiy in some way. 

14. A creative product in my field must ooirform to acceptable technical requirements. 


Personal Traits 

This seclian at the questionnaire asks you to evaluate tie extent to which each of 29 

statements relate to personal traits creative people in your field passes. Please fit n 

tie rK*nber(f = not important, 7 = extremef/ irryiorlanf) you feel best reflects how 

essential these statements are to personal traits of create people in your sul^ect area. 

1. A creative person in my field can generate a large number of ideas that are relevant 
to the problem at hand. 

2. Creative people in my subject area have an abiity to produrie uncccnmrm or unique 
responses. 

3. The ability to develop and elaborate upon ideas is a trait that creative people in my 
field possess. 

4. A creative person considers a variety of types of information when thinking about 
a problem. 

5. Beng open minded is an important trait one must prrssess to be rxmsidered 
creative in my field. 

6. When faced with a problem or chalenge a creative person is able to distinguish 
clearly between relevant and irrelevant information. 

7. The ability to resist the impulse to accept the first solution that comes to mind and 
to explore all possible ideas would be a trait of a creative person in my field. 

8. Creative people in my subject area tap into their emotions in order to generate 
ideas or solutions to a problem or challenge. 

9. A creative person has the abiity to put together ideas into novel artd pleasing 
comb nations. 

10. A creative person has the abiity to farrtasize and go beyond concrete reaity. 

11. A creative person seeks out ways to stimulate more than one of their senses to 
increase their flow of ideas. 

12. A creative individual may look at everyday objects ar>d see something rrovel and 
exciting. 

13. A creative person in my field is interested in looking beyond exteriors; exploring 
the imer workings of an object, problem or idea. 

14. The degree to which a person is able to look past the task at hand and visualize 
the systems it hnctions in is a charai4eristic of a creative person. 

15. Creative people in my field have the ability to see peculiarity ar>d have the ability to 
combme ideas or images in unusual ways that evoke surprise. 

16. In my field the ability to be humorous or playful is an indicator of a creative person. 

17. A key component of a creative persrxi in my field is a concern for the future, and a 
desire to be a part of its shaping. 

18. Creative people in my field usually show unusual interest in their particular 
pursuit 

19. Creative people in my field have great tolerance for vagueness. 

20. Creative people in my field possess great ktrowledge of the principles and theories 
relating to their area of interest 

21. Creative people in my field are known for the persistence that they bring to their 
work. 

22. Creative people in my field have the ability to improvise. 
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Personal Traits (continued) 


23. Creativity in my field is really a gift that cannot be taught. 

24. Creative people are seldom satisfied with their work and would rather not bring 
quick closure to a task. 

25. Creative people display flexibility of mind; they are capable of changing their 
mental set easily. 

26. Creative people can manipulate many related ideas at the same time. 

27. Creative people possess high intrinsic motivation for their work. 

28. Creative people tend to have novel ideas relating to their subject. 

29. Creative people have high sensitivity to problems; they can see challenges in 
situations where others are oblivious to them. 
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Environment 

This section of the questionnaire asks you to evaluate the extent to which each of IS 

statements relate to the role environment plays in promoting creativity in your field. 

Please circle the number ( f = not importani, 7 = extremely importanf) you feel best 

reflects the degree in which environment contributes to creativity in your subject area. 

1 . Creativity is enhanced when people work in groups. 

2. Creativity is enhanced in environments that allow risk taking. 

3. Creative people in my field tend to be more productive when they work by 
themselves. 

4. In my field work environments that are open and offer flexibility are aids to creative 
work. 

5. Classrooms that offer structure can be beneficial to the development of creativity. 

6. An atmosphere of competition tends to have a positive effect on creative work. 

7. Creativity is aided in environments that offer rewards for such work. 

8. Creativity is aided in environments that offer feedback about a person's work. 

9. Beir>g able to work within constraints is a measure of creativity. 

10. Creativity is fostered when people are encouraged to pursue activHies that are of 
irrterest to them. 

1 1 . Creativity is more likely to be achieved when one's activities are aimed towards a 
goal. 

12. Creative environments are usually messy or chaotic. 

13. In the classroom creativity is aided if the teacher provides guidelines for how the 
work should proceed. 

14. In the classroom students are more likely to be creative when the teacher allows 
them freedom to work in their own way. 

15. Students are more likely to produce creative work if they receive sound instruction 
in the krrowledge and principles relating to their work. 
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Appendix C Mean Comparison of the Creative Process Items 

Art 

Items/Statements Mean SD 

Q6 Having relevant knowledge of prior 5.39 1 .29 

products or solutions is an 
important aspect of creative work. 

Q7 To produce creative work a person 4.57 1 .60 

must be familiar with standards for 
acceptable solutions. 

Q8 The creative process requires the 6.32 .83 

ability to generate a number of 
exploratory ideas or solutions. 

Q9 Finding or identifying challenging 5.61 1.26 

problems is a critical dimension of 
the creative process 

Q10 Creativity includes the ability to 5.57 1.14 

find gaps, inconsistencies or 
flaws in existing solutions. 

Q11 Generating a representation of the 5.33 1.10 

problem or challenge is part of the 
creative process. 

Q12 Seeking out reactions to possible 5.16 1.34 

solutions is an important dimension 
of the creative process. 

Q13 The creative process sometimes 6.29 .90 

requires taking a break from the 
problem or challenge at hand to 
allow ideas to incubate. 


N Art = 75, n Music = 127, n Technology =42, *sig. p 


Music Technology 


Mean 

SD 

Mean 

SD 

P 

5.38 

1.16 

5.26 

1.45 

.856 

5.00 

1.49 

4.95 

1.43 

.143 

6.02 

.93 

6.10 

1.10 

.082 

5.37 

1.34 

5.19 

1.25 

.206 

5.13 

1.38 

5.26 

1.33 

.070 

4.88 

1.32 

5.26 

1.08 

.024 

4.94 

1.52 

5.14 

1.20 

.487 


5.99 


1.18 


5.38 


1.41 


.000 






Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q14 Creative solutions sometimes 
come to mind as a "flash" or 
sudden awareness 

6.23 

.97 

6.08 

1.08 

5.79 

1.07 

.093 

Q15 Metaphors and analogies are 
useful aids in creative thinking. 

5.93 

1.26 

5.86 

1.08 

5.33 

1.05 

.015* 

Q16 The act of creating sometimes 
involves reformulation of the initial 
problem or challenge as one 
becomes engaged in the work. 

6.08 

.88 

5.57 

1.04 

5.62 

.96 

.001* 

Q17 The creative process may begin 
even though the final product may 
not be formed in the "mind's eye". 

6.32 

.83 

6.06 

.98 

5.76 

.91 

.007* 

Q18 The creative process often 

includes gathering and drawing 
upon all resources that can be 
helpful in completing a task. 

6.08 

1.00 

5.81 

1.03 

6.02 

1.00 

.151 

Q19 The possession of relevant 
knowledge is an important aid 
to the creative process. 

5.76 

1.05 

5.82 

1.11 

5.52 

1.09 

.319 

Q20 Creativity is improved if a person 
that is familiar with technical rules. 

4.93 

1.48 

4.79 

1.57 

4.83 

1.64 

.823 

Q21 Creativity is improved if a person 
is familiar with relevant principles 
or theories. 

5.04 

1.34 

5.08 

1.40 

5.31 

1.12 

.548 


N Art = 75, n Music = 127, n Technology =42, 
*sig. p 
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Appendix D Mean Comparison of the Creative Product Items 


Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q22 In my field, a creative product must 
posses a high degree of novelty. 

4.64 

1.49 

3.95 

1.51 

4.10 

1.69 

.009* 

Q23 A creative product is likely to 
influence or suggest additional 
future creative products. 

5.67 

1.16 

5.06 

1.28 

5.07 

1.35 

.003* 

Q24 A product is considered creative if 
it is unusual or seen infrequently in 
the category to which it belongs. 

4.88 

1.65 

4.13 

1.44 

4.24 

1.46 

.003* 

Q25 The degree to which a product 
responds to a need or problem 
determines its level of creativity. 

4.05 

1.58 

3.57 

1.59 

4.74 

1.29 

.000* 

Q26 A creative product follows the 
accepted and understood rules 
of the discipline. 

3.48 

1.48 

3.55 

1.63 

4.00 

1.25 

.183 

Q27 A creative product has clear and 
practical implications. 

2.95 

1.43 

3.63 

1.69 

5.00 

1.23 

.000* 

Q28 To be considered creative, a 
product in my field must be of 
value to the community at large. 

2.72 

1.43 

3.31 

1.73 

4.60 

1.27 

.000* 

Q29 A creative product breaks with the 
tradition from which it emerges. 

4.55 

1.79 

3.92 

1.50 

4.93 

1.35 

.000* 


N Art = 75, n Music = 127, n Technology =42 
*sig. p 
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Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q30 Products are creative if they 
combine elements in unusual 
ways. 

5.57 

1.25 

5.01 

1.17 

4.98 

1.24 

.003* 

Q31 A product Is creative if it 

commands the attention of a 
person using, listening to, or 
viewing it. 

5.25 

1.41 

5.18 

1.41 

4.86 

1.35 

.316 

Q32 A creative product in my field Is 
easy to understand. Interpret, or 
use. 

3.19 

1.52 

4.07 

1.47 

4.93 

1.22 

.000* 

Q33 The craft component of completed 
works Is critical in determining how 
creative they are. 

3.49 

1.56 

4.11 

1.38 

4.45 

1.19 

.001* 

Q34 To be deemed creative, a product 
in my field must be revolutionary 
In some way. 

4.00 

1.69 

3.53 

1.47 

4.24 

1.30 

.014* 

Q35 A creative product In my field must 
conform to acceptable technical 
requirements. 

3.24 

1.45 

3.54 

1.67 

4.88 

1.35 

.000* 


N Art = 75, n Music = 127, n Technology =42 
*sig. p 
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Appendix E Mean Comparison of the Personal Trait Items 


Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q36 A creative person in my fieid can generate a 
iarge number of ideas that are reievant to the 
probiem at hand. 

5.71 

1.28 

5.21 

1.28 

5.57 

1.15 

.020 

Q37 Creative peopie in my subject area have an 
ability to produce uncommon or unique 
responses. 

6.15 

.82 

5.70 

.93 

5.57 

1.17 

.001* 

Q38 The ability to develop and elaborate upon 
ideas is a trait that creative people in my field 
possess. 

6.21 

.91 

5.94 

.98 

5.90 

1.10 

.112 

Q39 A creative person considers a variety of types 
of information when thinking about a problem. 

6.25 

.90 

5.74 

1.11 

6.05 

.94 

.002* 

Q40 Being open minded is an important trait one 
must possess to be considered creative in my 
field. 

6.33 

1.06 

5.87 

1.36 

6.29 

.74 

.013* 

Q41 When faced with a problem or challenge a 
creative person is able to distinguish clearly 
between relevant and irrelevant information. 

4.99 

1.34 

5.16 

1.30 

5.45 

.97 

.163 

Q42 The ability to resist the impulse to accept 
the first solution that comes to mind and to 
explore all possible ideas would be a trait 
of a creative person in my field. 

5.71 

1.39 

5.14 

1.43 

5.67 

1.28 

.010* 

Q43 Creative people in my subject area tap into 
their emotions in order to generate ideas or 
solutions to a problem or challenge. 

5.81 

1.01 

6.05 

.995 

3.86 

1.59 

.000* 

Q44 A creative person has the ability to put 
together ideas into novel and pleasing 
combinations. 

6.00 

.97 

5.85 

1.10 

5.00 

1.36 

.000* 


N Art = 75, n Music = 127, n Technology =42 
*sig. p 
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Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q45 A creative person has the ability to fantasize 
and go beyond concrete reality. 

6.21 

.87 

5.99 

1.18 

5.64 

.96 

.021 

Q46 A creative person seeks out ways to stimulate 
more than one of their senses to increase 
their flow of ideas. 

5.88 

1.12 

5.75 

1.27 

5.05 

1.38 

.002* 

Q47 A creative individual may look at everyday 
objects and see something novel and 
exciting. 

6.45 

.70 

5.85 

1.22 

5.79 

1.03 

.000* 

Q48 A creative person in my field is interested in 
looking beyond exteriors; exploring the inner 
workings of an object, problem or idea. 

6.07 

.95 

5.70 

1.25 

5.81 

1.09 

.093 

Q49 The degree to which a person is able to 
look past the task at hand and visualize the 
systems it functions in is a characteristic 
of a creative person. 

5.68 

.99 

5.28 

1.32 

5.62 

1.08 

.045 

Q50 Creative people in my field have the ability 
to see peculiarity and have the ability to 
combine ideas or images in unusual ways 
that evoke surprise. 

6.16 

.87 

5.58 

1.19 

5.31 

1.07 

.000* 

Q51 In my field the ability to be humorous or 
playful is an indicator of a creative person. 

5.03 

1.57 

5.34 

1.46 

4.29 

1.66 

.001* 

Q52 A key component of a creative person in my 
field is a concern for the future, and a desire 
to be a part of its shaping. 

4.69 

1.48 

4.82 

1.61 

5.40 

1.23 

.042 

Q53 Creative people in my field usually show 
unusual interest in their particular pursuit. 

5.49 

1.29 

5.94 

1.11 

5.52 

1.13 

.014* 


N Art = 75, n Music = 127, n Technology =42 
*sig. p 
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Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q54 Creative people in my field have great 
tolerance for vagueness. 

4.48 

1.66 

3.84 

1.56 

4.10 

1.78 

.028 

Q55 Creative people in my field possess great 
knowledge of the principles and theories 
relating to their area of interest. 

5.37 

1.30 

5.38 

1.29 

5.55 

1.06 

.733 

Q56 Creative people in my field are known for the 
persistence that they bring to their work. 

5.79 

1.20 

5.92 

1.21 

5.81 

.80 

.692 

Q57 Creative people in my field have the ability to 
improvise. 

6.43 

.81 

6.48 

.87 

6.36 

.73 

.675 

Q58 Creativity in my field is really a gift that cannot 
be taught. 

3.73 

1.70 

4.38 

1.70 

3.71 

1.73 

.013* 

Q59 Creative people are seldom satisfied with 
their work and would rather not bring quick 
closure to a task. 

4.61 

1.43 

4.87 

1.42 

4.55 

1.42 

.315 

Q60 Creative people display flexibility of mind; 
they are capable of changing their mental set 
easily. 

5.45 

1.15 

5.06 

1.17 

4.86 

1.34 

.018* 

Q61 Creative people can manipulate many related 
ideas at the same time. 

5.71 

1.04 

5.48 

1.19 

5.19 

1.33 

.072 

Q62 Creative people possess high intrinsic 
motivation for their work. 

5.95 

1.10 

6.00 

1.31 

5.55 

.99 

.066 

Q63 Creative people tend to have novel ideas 
relating to their subject. 

5.81 

1.15 

5.78 

1.13 

5.38 

.854 

.087 

Q64 Creative people have high sensitivity to 
problems: they can see challenges in 
situations where others are oblivious to them. 

5.64 

1.07 

5.56 

1.13 

5.64 

.96 

.856 


N Art = 75, n Music = 127, n Technology =42 
*sig. p 
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Appendix F Mean Comparison of the Creative Environment Items 


Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q65 Creativity is enhanced when peopie work in 
groups. 

4.03 

1.66 

4.45 

1.71 

5.14 

1.42 

.002* 

Q66 Creativity is enhanced in environments that 
aiiow risk taking. 

6.41 

.89 

6.16 

1.14 

6.07 

.95 

.144 

Q67 Creative peopie in my fieid tend to be more 
productive when they work by themseives. 

4.63 

1.45 

4.39 

1.51 

4.10 

1.61 

.186 

Q68 In my fieid work environments that are open 
and offer fiexibiiity are aids to creative work. 

6.05 

1.13 

5.74 

1.26 

5.60 

1.28 

.105 

Q69 Ciassrooms that offer structure can be 
beneficiai to the deveiopment of creativity. 

5.12 

1.40 

4.97 

1.39 

4.93 

1.26 

.688 

Q70 An atmosphere of competition tends to have 
a positive effect on creative work. 

3.97 

1.55 

3.75 

1.63 

5.14 

1.34 

.000* 

Q71 Creativity is aided in environments that offer 
rewards for such work. 

4.17 

1.54 

4.37 

1.52 

5.55 

.92 

.000* 

Q72 Creativity is aided in environments that offer 
feedback about a person’s work. 

5.83 

1.19 

5.48 

1.24 

5.64 

1.01 

.137 

Q73 Being abie to work within constraints is a 
measure of creativity. 

4.69 

1.78 

4.53 

1.72 

4.83 

1.36 

.559 

Q74 Creativity is fostered when peopie are 
encouraged to pursue activities that are of 
interest to them. 

6.09 

1.04 

6.25 

.89 

5.95 

.85 

.172 


N Art = 75, n Music = 127, n Technology =42 
*sig. p 
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Items/Statements 

Art 

Mean 

SD 

Music 

Mean 

SD 

Technology 
Mean SD 

P 

Q75 Creativity is more likely to be achieved when 
one’s activities are aimed towards a goal. 

5.24 

1.21 

5.64 

1.14 

5.69 

1.07 

.034 

Q76 Creative environments are usually messy or 
chaotic. 

3.63 

1.60 

3.90 

1.61 

4.00 

1.56 

.377 

Q77 In the classroom creativity is aided if the 

teacher provides guidelines for how the work 
should proceed. 

5.01 

1.36 

5.09 

1.30 

4.64 

1.27 

.163 

Q78 In the classroom students are more likely to 
be creative when the teacher allows them 
freedom to work In their own way. 

5.12 

1.30 

5.14 

1.26 

5.26 

1.23 

.832 

Q79 Students are more likely to produce creative 
work if they receive sound instruction in the 
knowledge and principles relating to their 
work. 

5.65 

1.16 

5.78 

1.18 

5.43 

1.25 

.250 


N Art = 75, n Music = 127, n Technology =42 
*sig p 
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